2024-06-18T12:45:34.052 INFO:root:teuthology version: 0.0.1.dev325+g8e97141 2024-06-18T12:45:34.053 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-06-17_21:08:03-orch-squid-distro-default-smithi/7760537 --description orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} --name teuthology-2024-06-17_21:08:03-orch-squid-distro-default-smithi --owner scheduled_teuthology@teuthology --verbose -- /home/teuthworker/archive/teuthology-2024-06-17_21:08:03-orch-squid-distro-default-smithi/7760537/orig.config.yaml 2024-06-18T12:45:34.080 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-18T12:45:34.161 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-17_21:08:03-orch-squid-distro-default-smithi/7760537 branch: squid description: orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} email: ceph-qa@ceph.com first_in_suite: false job_id: '7760537' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-06-17_21:08:03-orch-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) log-only-match: - CEPHADM_ sha1: 3781e3a592387d16e8020f5db99cdee4198f6d00 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 3781e3a592387d16e8020f5db99cdee4198f6d00 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: squid sha1: f18f3f1cbd11e4a41c557e616d172f24deb91473 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4712 sha1: 3781e3a592387d16e8020f5db99cdee4198f6d00 sleep_before_teardown: 0 subset: 61/64 suite: orch suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_f18f3f1cbd11e4a41c557e616d172f24deb91473/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: f18f3f1cbd11e4a41c557e616d172f24deb91473 targets: smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKXq63vw6iXZUy8CCj6gVgLI7VTsxn661hKFvUal3Z/Hfd8H5jbdTzi4klg4Djwd6OqLscdUDxcUa/UFALAB9nQ= smithi179.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPY/9oKvTb3gi6a/L1V6Z6dPstH7cy1nW1EPlWlkx8J6Mki6mwNyCClTK2u1ChE/nhKpHLS6JsA58zSQI5wW4Gg= tasks: - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e9714173de9e92c97e8ef1045d333e96b793454 timestamp: 2024-06-17_21:08:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-18T12:45:34.162 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_f18f3f1cbd11e4a41c557e616d172f24deb91473/qa; will attempt to use it 2024-06-18T12:45:34.162 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_f18f3f1cbd11e4a41c557e616d172f24deb91473/qa/tasks 2024-06-18T12:45:34.163 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-18T12:45:34.164 INFO:teuthology.task.internal:Checking packages... 2024-06-18T12:45:34.189 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3781e3a592387d16e8020f5db99cdee4198f6d00' 2024-06-18T12:45:34.189 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-18T12:45:34.189 INFO:teuthology.packaging:ref: None 2024-06-18T12:45:34.189 INFO:teuthology.packaging:tag: None 2024-06-18T12:45:34.189 INFO:teuthology.packaging:branch: squid 2024-06-18T12:45:34.189 INFO:teuthology.packaging:sha1: 3781e3a592387d16e8020f5db99cdee4198f6d00 2024-06-18T12:45:34.190 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-18T12:45:34.393 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3145.gf18f3f1c 2024-06-18T12:45:34.395 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-18T12:45:34.401 INFO:teuthology.task.internal:no buildpackages task found 2024-06-18T12:45:34.401 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-18T12:45:34.421 INFO:teuthology.task.internal:Saving configuration 2024-06-18T12:45:34.431 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-18T12:45:34.439 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-18T12:45:34.463 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-17_21:08:03-orch-squid-distro-default-smithi/7760537', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-18 12:40:28.756688', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKXq63vw6iXZUy8CCj6gVgLI7VTsxn661hKFvUal3Z/Hfd8H5jbdTzi4klg4Djwd6OqLscdUDxcUa/UFALAB9nQ='} 2024-06-18T12:45:34.487 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-17_21:08:03-orch-squid-distro-default-smithi/7760537', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-18 12:40:28.758544', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPY/9oKvTb3gi6a/L1V6Z6dPstH7cy1nW1EPlWlkx8J6Mki6mwNyCClTK2u1ChE/nhKpHLS6JsA58zSQI5wW4Gg='} 2024-06-18T12:45:34.487 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-18T12:45:34.493 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-06-18T12:45:34.494 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-06-18T12:45:34.494 INFO:teuthology.run_tasks:Running task console_log... 2024-06-18T12:45:34.544 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3ce3923ac0>, signals=[15]) 2024-06-18T12:45:34.544 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-18T12:45:34.549 INFO:teuthology.task.internal:Opening connections... 2024-06-18T12:45:34.550 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-06-18T12:45:34.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-18T12:45:34.626 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2024-06-18T12:45:34.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-18T12:45:34.698 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-18T12:45:34.705 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-06-18T12:45:34.723 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-06-18T12:45:34.723 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-06-18T12:45:34.780 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-06-18T12:45:34.781 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-06-18T12:45:34.781 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-06-18T12:45:34.781 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-06-18T12:45:34.781 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-06-18T12:45:34.781 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-06-18T12:45:34.781 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-18T12:45:34.781 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-06-18T12:45:34.781 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-06-18T12:45:34.781 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-18T12:45:34.781 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-06-18T12:45:34.781 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-18T12:45:34.781 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-18T12:45:34.781 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-18T12:45:34.782 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-06-18T12:45:34.799 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2024-06-18T12:45:34.817 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2024-06-18T12:45:34.817 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2024-06-18T12:45:34.873 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream" 2024-06-18T12:45:34.873 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="9" 2024-06-18T12:45:34.873 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos" 2024-06-18T12:45:34.873 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora" 2024-06-18T12:45:34.874 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="9" 2024-06-18T12:45:34.874 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el9" 2024-06-18T12:45:34.874 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-18T12:45:34.874 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2024-06-18T12:45:34.874 INFO:teuthology.orchestra.run.smithi179.stdout:LOGO="fedora-logo-icon" 2024-06-18T12:45:34.874 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-18T12:45:34.874 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/" 2024-06-18T12:45:34.874 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-18T12:45:34.874 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-18T12:45:34.874 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-18T12:45:34.875 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2024-06-18T12:45:34.899 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-18T12:45:34.907 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-18T12:45:34.915 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-18T12:45:34.915 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-06-18T12:45:34.917 DEBUG:teuthology.orchestra.run.smithi179:> test '!' -e /home/ubuntu/cephtest 2024-06-18T12:45:34.933 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-18T12:45:34.939 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-18T12:45:34.939 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-06-18T12:45:34.974 DEBUG:teuthology.orchestra.run.smithi179:> test -z $(ls -A /var/lib/ceph) 2024-06-18T12:45:35.062 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-18T12:45:35.118 INFO:teuthology.run_tasks:Running task kernel... 2024-06-18T12:45:35.132 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-06-18T12:45:35.133 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-06-18T12:45:35.133 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-18T12:45:35.133 DEBUG:teuthology.orchestra.run.smithi179:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-18T12:45:35.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-18T12:45:35.150 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-06-18T12:45:35.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-18T12:45:35.151 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-06-18T12:45:35.207 INFO:teuthology.orchestra.run.smithi167.stdout:5.14.0-437.el9.x86_64 2024-06-18T12:45:35.207 INFO:teuthology.task.kernel:Running kernel on smithi167: 5.14.0-437.el9.x86_64 2024-06-18T12:45:35.208 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum install -y kernel 2024-06-18T12:45:35.209 INFO:teuthology.orchestra.run.smithi179.stdout:5.14.0-437.el9.x86_64 2024-06-18T12:45:35.209 INFO:teuthology.task.kernel:Running kernel on smithi179: 5.14.0-437.el9.x86_64 2024-06-18T12:45:35.209 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum install -y kernel 2024-06-18T12:45:37.921 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 8.4 MB/s | 8.1 MB 00:00 2024-06-18T12:45:39.888 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.1 MB 00:03 2024-06-18T12:45:41.363 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 20 MB 00:02 2024-06-18T12:45:43.197 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 20 MB 00:02 2024-06-18T12:45:45.822 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.3 MB 00:00 2024-06-18T12:45:47.498 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 17 kB 00:00 2024-06-18T12:45:48.290 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.3 MB 00:01 2024-06-18T12:45:48.868 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-06-18T12:45:49.707 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 17 kB 00:00 2024-06-18T12:45:51.003 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-06-18T12:45:54.157 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-06-18T12:45:56.142 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-06-18T12:45:56.189 INFO:teuthology.orchestra.run.smithi167.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-18T12:45:56.189 INFO:teuthology.orchestra.run.smithi167.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-18T12:45:56.278 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:45:56.288 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:45:56.288 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2024-06-18T12:45:56.288 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:45:56.288 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:45:56.288 INFO:teuthology.orchestra.run.smithi167.stdout: kernel x86_64 5.14.0-460.el9 baseos 1.1 M 2024-06-18T12:45:56.288 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-06-18T12:45:56.288 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-core x86_64 5.14.0-460.el9 baseos 16 M 2024-06-18T12:45:56.288 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-modules x86_64 5.14.0-460.el9 baseos 35 M 2024-06-18T12:45:56.288 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-modules-core x86_64 5.14.0-460.el9 baseos 29 M 2024-06-18T12:45:56.288 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:45:56.288 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:45:56.288 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:45:56.288 INFO:teuthology.orchestra.run.smithi167.stdout:Install 4 Packages 2024-06-18T12:45:56.289 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:45:56.289 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 81 M 2024-06-18T12:45:56.289 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 124 M 2024-06-18T12:45:56.289 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:45:57.680 INFO:teuthology.orchestra.run.smithi167.stdout:(1/4): kernel-5.14.0-460.el9.x86_64.rpm 864 kB/s | 1.1 MB 00:01 2024-06-18T12:45:58.174 INFO:teuthology.orchestra.run.smithi179.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-18T12:45:58.174 INFO:teuthology.orchestra.run.smithi179.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-18T12:45:58.269 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:45:58.275 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:45:58.275 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-06-18T12:45:58.275 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:45:58.275 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:45:58.276 INFO:teuthology.orchestra.run.smithi179.stdout: kernel x86_64 5.14.0-460.el9 baseos 1.1 M 2024-06-18T12:45:58.276 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-06-18T12:45:58.276 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-core x86_64 5.14.0-460.el9 baseos 16 M 2024-06-18T12:45:58.276 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules x86_64 5.14.0-460.el9 baseos 35 M 2024-06-18T12:45:58.276 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules-core x86_64 5.14.0-460.el9 baseos 29 M 2024-06-18T12:45:58.276 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:45:58.276 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:45:58.276 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:45:58.276 INFO:teuthology.orchestra.run.smithi179.stdout:Install 4 Packages 2024-06-18T12:45:58.276 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:45:58.277 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 81 M 2024-06-18T12:45:58.277 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 124 M 2024-06-18T12:45:58.277 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:45:58.392 INFO:teuthology.orchestra.run.smithi167.stdout:(2/4): kernel-core-5.14.0-460.el9.x86_64.rpm 8.2 MB/s | 16 MB 00:01 2024-06-18T12:45:59.318 INFO:teuthology.orchestra.run.smithi167.stdout:(3/4): kernel-modules-5.14.0-460.el9.x86_64.rpm 12 MB/s | 35 MB 00:02 2024-06-18T12:45:59.661 INFO:teuthology.orchestra.run.smithi179.stdout:(1/4): kernel-5.14.0-460.el9.x86_64.rpm 969 kB/s | 1.1 MB 00:01 2024-06-18T12:46:00.276 INFO:teuthology.orchestra.run.smithi167.stdout:(4/4): kernel-modules-core-5.14.0-460.el9.x86_6 11 MB/s | 29 MB 00:02 2024-06-18T12:46:00.276 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:46:00.277 INFO:teuthology.orchestra.run.smithi167.stdout:Total 20 MB/s | 81 MB 00:03 2024-06-18T12:46:00.977 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:46:01.336 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:46:01.336 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:46:02.302 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:46:02.302 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:46:04.717 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:46:05.354 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : kernel-modules-core-5.14.0-460.el9.x86_64 1/4 2024-06-18T12:46:05.444 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : kernel-core-5.14.0-460.el9.x86_64 2/4 2024-06-18T12:46:08.121 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-core-5.14.0-460.el9.x86_64 2/4 2024-06-18T12:46:08.267 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : kernel-modules-5.14.0-460.el9.x86_64 3/4 2024-06-18T12:46:13.042 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-modules-5.14.0-460.el9.x86_64 3/4 2024-06-18T12:46:13.149 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : kernel-5.14.0-460.el9.x86_64 4/4 2024-06-18T12:46:13.602 INFO:teuthology.orchestra.run.smithi179.stdout:(2/4): kernel-core-5.14.0-460.el9.x86_64.rpm 1.1 MB/s | 16 MB 00:15 2024-06-18T12:46:17.910 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-modules-core-5.14.0-460.el9.x86_64 4/4 2024-06-18T12:46:30.467 INFO:teuthology.orchestra.run.smithi179.stdout:(3/4): kernel-modules-core-5.14.0-460.el9.x86_6 967 kB/s | 29 MB 00:30 2024-06-18T12:46:39.416 INFO:teuthology.orchestra.run.smithi179.stdout:(4/4): kernel-modules-5.14.0-460.el9.x86_64.rpm 872 kB/s | 35 MB 00:40 2024-06-18T12:46:39.416 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:46:39.416 INFO:teuthology.orchestra.run.smithi179.stdout:Total 2.0 MB/s | 81 MB 00:41 2024-06-18T12:46:40.118 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:46:40.469 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:46:40.469 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:46:41.407 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:46:41.407 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:46:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:46:44.579 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-modules-core-5.14.0-460.el9.x86_64 1/4 2024-06-18T12:46:44.669 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-core-5.14.0-460.el9.x86_64 2/4 2024-06-18T12:46:47.323 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-core-5.14.0-460.el9.x86_64 2/4 2024-06-18T12:46:47.478 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-modules-5.14.0-460.el9.x86_64 3/4 2024-06-18T12:46:47.609 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-core-5.14.0-460.el9.x86_64 4/4 2024-06-18T12:46:47.650 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-modules-5.14.0-460.el9.x86_64 4/4 2024-06-18T12:46:47.939 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-5.14.0-460.el9.x86_64 4/4 2024-06-18T12:46:47.940 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-5.14.0-460.el9.x86_64 1/4 2024-06-18T12:46:47.940 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-core-5.14.0-460.el9.x86_64 2/4 2024-06-18T12:46:47.940 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-modules-5.14.0-460.el9.x86_64 3/4 2024-06-18T12:46:48.756 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-modules-core-5.14.0-460.el9.x86_64 4/4 2024-06-18T12:46:48.756 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:46:48.757 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:46:48.757 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-5.14.0-460.el9.x86_64 2024-06-18T12:46:48.757 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-core-5.14.0-460.el9.x86_64 2024-06-18T12:46:48.757 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-modules-5.14.0-460.el9.x86_64 2024-06-18T12:46:48.757 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-modules-core-5.14.0-460.el9.x86_64 2024-06-18T12:46:48.757 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:46:48.757 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:46:49.113 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q kernel | sort -rV | head -n 1 2024-06-18T12:46:49.195 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-5.14.0-460.el9.x86_64 2024-06-18T12:46:49.195 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-460.el9.x86_64 2024-06-18T12:46:49.195 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-460.el9.x86_64 2024-06-18T12:46:49.195 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-18T12:46:49.195 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-18T12:46:49.196 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-06-18T12:46:49.196 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi167.front.sepia.ceph.com, path=None, version=distro) 2024-06-18T12:46:49.196 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q kernel | sort -rV | head -n 1 2024-06-18T12:46:49.249 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-5.14.0-460.el9.x86_64 2024-06-18T12:46:49.249 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-460.el9.x86_64 2024-06-18T12:46:49.249 DEBUG:teuthology.orchestra.run.smithi167:> sudo rpm -qi grub2-tools 2024-06-18T12:46:49.289 INFO:teuthology.orchestra.run.smithi167.stdout:Name : grub2-tools 2024-06-18T12:46:49.289 INFO:teuthology.orchestra.run.smithi167.stdout:Epoch : 1 2024-06-18T12:46:49.290 INFO:teuthology.orchestra.run.smithi167.stdout:Version : 2.06 2024-06-18T12:46:49.290 INFO:teuthology.orchestra.run.smithi167.stdout:Release : 68.el9 2024-06-18T12:46:49.290 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: x86_64 2024-06-18T12:46:49.290 INFO:teuthology.orchestra.run.smithi167.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-18T12:46:49.290 INFO:teuthology.orchestra.run.smithi167.stdout:Group : Unspecified 2024-06-18T12:46:49.290 INFO:teuthology.orchestra.run.smithi167.stdout:Size : 8264254 2024-06-18T12:46:49.290 INFO:teuthology.orchestra.run.smithi167.stdout:License : GPLv3+ 2024-06-18T12:46:49.290 INFO:teuthology.orchestra.run.smithi167.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-18T12:46:49.290 INFO:teuthology.orchestra.run.smithi167.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-18T12:46:49.290 INFO:teuthology.orchestra.run.smithi167.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-18T12:46:49.290 INFO:teuthology.orchestra.run.smithi167.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-18T12:46:49.290 INFO:teuthology.orchestra.run.smithi167.stdout:Packager : builder@centos.org 2024-06-18T12:46:49.290 INFO:teuthology.orchestra.run.smithi167.stdout:Vendor : CentOS 2024-06-18T12:46:49.290 INFO:teuthology.orchestra.run.smithi167.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-18T12:46:49.291 INFO:teuthology.orchestra.run.smithi167.stdout:Summary : Support tools for GRUB. 2024-06-18T12:46:49.291 INFO:teuthology.orchestra.run.smithi167.stdout:Description : 2024-06-18T12:46:49.291 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:46:49.291 INFO:teuthology.orchestra.run.smithi167.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-18T12:46:49.291 INFO:teuthology.orchestra.run.smithi167.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-18T12:46:49.291 INFO:teuthology.orchestra.run.smithi167.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-18T12:46:49.291 INFO:teuthology.orchestra.run.smithi167.stdout:hardware devices. 2024-06-18T12:46:49.291 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:46:49.291 INFO:teuthology.orchestra.run.smithi167.stdout:This subpackage provides tools for support of all platforms. 2024-06-18T12:46:49.292 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-18T12:46:49.292 INFO:teuthology.task.kernel:Updating grub on smithi167 to boot 5.14.0-460.el9.x86_64 2024-06-18T12:46:49.292 DEBUG:teuthology.orchestra.run.smithi167:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-18T12:46:49.899 INFO:teuthology.orchestra.run.smithi167.stderr:Generating grub configuration file ... 2024-06-18T12:46:51.284 INFO:teuthology.orchestra.run.smithi167.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-18T12:46:51.315 INFO:teuthology.orchestra.run.smithi167.stderr:done 2024-06-18T12:46:51.320 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2024-06-18T12:46:51.337 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.1Ro4xxUsPC 2024-06-18T12:46:51.337 DEBUG:teuthology.orchestra.run.smithi167:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1Ro4xxUsPC 2024-06-18T12:46:51.403 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.1Ro4xxUsPC 2024-06-18T12:46:51.544 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.1Ro4xxUsPC is 6KB 2024-06-18T12:46:51.593 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.1Ro4xxUsPC 2024-06-18T12:46:51.607 DEBUG:teuthology.orchestra.run.smithi167:> sudo /bin/ls /boot/loader/entries || true 2024-06-18T12:46:51.679 INFO:teuthology.orchestra.run.smithi167.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-18T12:46:51.679 INFO:teuthology.orchestra.run.smithi167.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-18T12:46:51.679 INFO:teuthology.orchestra.run.smithi167.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-18T12:46:51.679 INFO:teuthology.orchestra.run.smithi167.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-460.el9.x86_64.conf 2024-06-18T12:46:51.681 DEBUG:teuthology.orchestra.run.smithi167:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-460.el9.x86_64 2024-06-18T12:46:51.825 DEBUG:teuthology.orchestra.run.smithi167:> sudo shutdown -r now 2024-06-18T12:46:52.231 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-modules-5.14.0-460.el9.x86_64 3/4 2024-06-18T12:46:52.376 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-5.14.0-460.el9.x86_64 4/4 2024-06-18T12:46:57.123 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-modules-core-5.14.0-460.el9.x86_64 4/4 2024-06-18T12:47:21.828 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-06-18T12:47:21.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-18T12:47:21.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-18T12:47:26.935 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-core-5.14.0-460.el9.x86_64 4/4 2024-06-18T12:47:26.973 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-modules-5.14.0-460.el9.x86_64 4/4 2024-06-18T12:47:27.321 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-5.14.0-460.el9.x86_64 4/4 2024-06-18T12:47:27.321 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-5.14.0-460.el9.x86_64 1/4 2024-06-18T12:47:27.321 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-core-5.14.0-460.el9.x86_64 2/4 2024-06-18T12:47:27.321 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-modules-5.14.0-460.el9.x86_64 3/4 2024-06-18T12:47:28.101 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-modules-core-5.14.0-460.el9.x86_64 4/4 2024-06-18T12:47:28.101 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:47:28.101 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:47:28.101 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-5.14.0-460.el9.x86_64 2024-06-18T12:47:28.101 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-core-5.14.0-460.el9.x86_64 2024-06-18T12:47:28.101 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules-5.14.0-460.el9.x86_64 2024-06-18T12:47:28.101 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules-core-5.14.0-460.el9.x86_64 2024-06-18T12:47:28.101 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:47:28.101 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:47:28.507 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q kernel | sort -rV | head -n 1 2024-06-18T12:47:28.583 INFO:teuthology.orchestra.run.smithi179.stdout:kernel-5.14.0-460.el9.x86_64 2024-06-18T12:47:28.583 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-460.el9.x86_64 2024-06-18T12:47:28.583 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-460.el9.x86_64 2024-06-18T12:47:28.583 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-18T12:47:28.583 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-18T12:47:28.583 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-06-18T12:47:28.584 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi179.front.sepia.ceph.com, path=None, version=distro) 2024-06-18T12:47:28.584 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q kernel | sort -rV | head -n 1 2024-06-18T12:47:28.637 INFO:teuthology.orchestra.run.smithi179.stdout:kernel-5.14.0-460.el9.x86_64 2024-06-18T12:47:28.637 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-460.el9.x86_64 2024-06-18T12:47:28.637 DEBUG:teuthology.orchestra.run.smithi179:> sudo rpm -qi grub2-tools 2024-06-18T12:47:28.680 INFO:teuthology.orchestra.run.smithi179.stdout:Name : grub2-tools 2024-06-18T12:47:28.680 INFO:teuthology.orchestra.run.smithi179.stdout:Epoch : 1 2024-06-18T12:47:28.680 INFO:teuthology.orchestra.run.smithi179.stdout:Version : 2.06 2024-06-18T12:47:28.680 INFO:teuthology.orchestra.run.smithi179.stdout:Release : 68.el9 2024-06-18T12:47:28.680 INFO:teuthology.orchestra.run.smithi179.stdout:Architecture: x86_64 2024-06-18T12:47:28.681 INFO:teuthology.orchestra.run.smithi179.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-18T12:47:28.681 INFO:teuthology.orchestra.run.smithi179.stdout:Group : Unspecified 2024-06-18T12:47:28.681 INFO:teuthology.orchestra.run.smithi179.stdout:Size : 8264254 2024-06-18T12:47:28.681 INFO:teuthology.orchestra.run.smithi179.stdout:License : GPLv3+ 2024-06-18T12:47:28.681 INFO:teuthology.orchestra.run.smithi179.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-18T12:47:28.681 INFO:teuthology.orchestra.run.smithi179.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-18T12:47:28.681 INFO:teuthology.orchestra.run.smithi179.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-18T12:47:28.681 INFO:teuthology.orchestra.run.smithi179.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-18T12:47:28.681 INFO:teuthology.orchestra.run.smithi179.stdout:Packager : builder@centos.org 2024-06-18T12:47:28.681 INFO:teuthology.orchestra.run.smithi179.stdout:Vendor : CentOS 2024-06-18T12:47:28.682 INFO:teuthology.orchestra.run.smithi179.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-18T12:47:28.682 INFO:teuthology.orchestra.run.smithi179.stdout:Summary : Support tools for GRUB. 2024-06-18T12:47:28.682 INFO:teuthology.orchestra.run.smithi179.stdout:Description : 2024-06-18T12:47:28.682 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:47:28.682 INFO:teuthology.orchestra.run.smithi179.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-18T12:47:28.682 INFO:teuthology.orchestra.run.smithi179.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-18T12:47:28.682 INFO:teuthology.orchestra.run.smithi179.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-18T12:47:28.682 INFO:teuthology.orchestra.run.smithi179.stdout:hardware devices. 2024-06-18T12:47:28.682 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:47:28.682 INFO:teuthology.orchestra.run.smithi179.stdout:This subpackage provides tools for support of all platforms. 2024-06-18T12:47:28.683 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-18T12:47:28.684 INFO:teuthology.task.kernel:Updating grub on smithi179 to boot 5.14.0-460.el9.x86_64 2024-06-18T12:47:28.684 DEBUG:teuthology.orchestra.run.smithi179:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-18T12:47:29.313 INFO:teuthology.orchestra.run.smithi179.stderr:Generating grub configuration file ... 2024-06-18T12:47:30.720 INFO:teuthology.orchestra.run.smithi179.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-18T12:47:30.754 INFO:teuthology.orchestra.run.smithi179.stderr:done 2024-06-18T12:47:30.756 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2024-06-18T12:47:30.771 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.iJmHqAgqQz 2024-06-18T12:47:30.771 DEBUG:teuthology.orchestra.run.smithi179:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.iJmHqAgqQz 2024-06-18T12:47:30.845 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.iJmHqAgqQz 2024-06-18T12:47:31.002 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.iJmHqAgqQz is 6KB 2024-06-18T12:47:31.051 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.iJmHqAgqQz 2024-06-18T12:47:31.065 DEBUG:teuthology.orchestra.run.smithi179:> sudo /bin/ls /boot/loader/entries || true 2024-06-18T12:47:31.126 INFO:teuthology.orchestra.run.smithi179.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-18T12:47:31.126 INFO:teuthology.orchestra.run.smithi179.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-18T12:47:31.126 INFO:teuthology.orchestra.run.smithi179.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-18T12:47:31.126 INFO:teuthology.orchestra.run.smithi179.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-460.el9.x86_64.conf 2024-06-18T12:47:31.127 DEBUG:teuthology.orchestra.run.smithi179:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-460.el9.x86_64 2024-06-18T12:47:31.275 DEBUG:teuthology.orchestra.run.smithi179:> sudo shutdown -r now 2024-06-18T12:47:40.319 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-06-18T12:47:49.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-18T12:47:49.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-18T12:47:52.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-06-18T12:48:01.286 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-06-18T12:48:01.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-06-18T12:48:01.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-18T12:48:04.386 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-18T12:48:04.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-18T12:48:19.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.179 2024-06-18T12:48:22.818 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-06-18T12:48:28.750 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-06-18T12:48:28.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-18T12:48:37.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-18T12:48:37.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-18T12:48:40.265 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-06-18T12:48:40.801 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-18T12:48:40.802 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-460.el9.x86_64"... 2024-06-18T12:48:40.802 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-06-18T12:48:40.816 INFO:teuthology.orchestra.run.smithi167.stdout:5.14.0-460.el9.x86_64 2024-06-18T12:48:40.816 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-460.el9.x86_64 vs 5.14.0-460.el9.x86_64 2024-06-18T12:48:40.816 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-18T12:48:40.816 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-18T12:48:41.817 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-06-18T12:48:41.817 DEBUG:teuthology.orchestra.run.smithi167:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-18T12:48:41.893 INFO:teuthology.orchestra.run.smithi167.stdout:ttyS1 2024-06-18T12:48:41.927 DEBUG:teuthology.parallel:result is None 2024-06-18T12:49:28.754 DEBUG:teuthology.orchestra.remote:timed out 2024-06-18T12:49:40.755 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-06-18T12:49:40.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-18T12:49:41.272 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-06-18T12:49:41.635 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-06-18T12:49:41.635 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-460.el9.x86_64"... 2024-06-18T12:49:41.635 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-06-18T12:49:41.651 INFO:teuthology.orchestra.run.smithi179.stdout:5.14.0-460.el9.x86_64 2024-06-18T12:49:41.651 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-460.el9.x86_64 vs 5.14.0-460.el9.x86_64 2024-06-18T12:49:41.651 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-18T12:49:41.651 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-18T12:49:42.652 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-06-18T12:49:42.652 DEBUG:teuthology.orchestra.run.smithi179:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-18T12:49:42.750 INFO:teuthology.orchestra.run.smithi179.stdout:ttyS1 2024-06-18T12:49:42.785 DEBUG:teuthology.parallel:result is None 2024-06-18T12:49:42.786 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-18T12:49:42.795 INFO:teuthology.task.internal:Creating test directory... 2024-06-18T12:49:42.795 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-18T12:49:42.799 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-18T12:49:42.817 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-18T12:49:42.855 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-18T12:49:42.917 INFO:teuthology.task.internal:Creating archive directory... 2024-06-18T12:49:42.917 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-18T12:49:42.920 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-18T12:49:43.036 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-18T12:49:43.047 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-18T12:49:43.047 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-18T12:49:43.050 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-18T12:49:43.081 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-18T12:49:43.092 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-18T12:49:43.093 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-18T12:49:43.103 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-18T12:49:43.105 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-18T12:49:43.117 INFO:teuthology.task.internal:Configuring sudo... 2024-06-18T12:49:43.117 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-18T12:49:43.137 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-18T12:49:43.172 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-18T12:49:43.185 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-18T12:49:43.185 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-18T12:49:43.205 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-18T12:49:43.229 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-18T12:49:43.303 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-18T12:49:43.351 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-18T12:49:43.432 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-18T12:49:43.475 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T12:49:43.475 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-18T12:49:43.540 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-18T12:49:43.584 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-18T12:49:43.635 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-18T12:49:43.718 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-18T12:49:43.762 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-18T12:49:43.762 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-18T12:49:43.829 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-06-18T12:49:43.831 DEBUG:teuthology.orchestra.run.smithi179:> sudo service rsyslog restart 2024-06-18T12:49:43.883 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-18T12:49:43.910 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-18T12:49:44.426 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-18T12:49:44.433 INFO:teuthology.task.internal:Starting timer... 2024-06-18T12:49:44.434 INFO:teuthology.run_tasks:Running task pcp... 2024-06-18T12:49:44.444 INFO:teuthology.run_tasks:Running task selinux... 2024-06-18T12:49:44.452 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-06-18T12:49:44.453 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2024-06-18T12:49:44.501 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2024-06-18T12:49:44.503 DEBUG:teuthology.orchestra.run.smithi179:> sudo service auditd rotate 2024-06-18T12:49:44.562 INFO:teuthology.orchestra.run.smithi179.stdout:Rotating logs: 2024-06-18T12:49:44.564 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-18T12:49:44.564 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2024-06-18T12:49:44.603 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2024-06-18T12:49:44.603 DEBUG:teuthology.orchestra.run.smithi179:> /usr/sbin/getenforce 2024-06-18T12:49:44.650 INFO:teuthology.orchestra.run.smithi179.stdout:Permissive 2024-06-18T12:49:44.651 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi167.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi179.front.sepia.ceph.com': 'permissive'} 2024-06-18T12:49:44.651 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-06-18T12:49:44.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-18T12:49:44.676 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-06-18T12:49:44.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-18T12:49:44.706 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-18T12:49:44.707 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2024-06-18T12:49:44.741 DEBUG:teuthology.orchestra.run.smithi179:> sudo /usr/sbin/setenforce permissive 2024-06-18T12:49:44.778 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-18T12:49:44.788 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-18T12:49:44.791 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-18T12:49:44.932 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-18T12:49:44.942 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-18T12:49:44.943 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi167.front.sepia.ceph.com,smithi179.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-18T12:54:05.942 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi167.front.sepia.ceph.com'), Remote(name='ubuntu@smithi179.front.sepia.ceph.com')] 2024-06-18T12:54:05.944 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-18T12:54:05.945 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-18T12:54:06.023 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-06-18T12:54:06.106 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-18T12:54:06.107 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-06-18T12:54:06.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-18T12:54:06.180 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-06-18T12:54:06.259 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-06-18T12:54:06.259 INFO:teuthology.run_tasks:Running task clock... 2024-06-18T12:54:06.269 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-18T12:54:06.270 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-18T12:54:06.270 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-18T12:54:06.273 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-18T12:54:06.273 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-18T12:54:06.299 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-18T12:54:06.314 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-18T12:54:06.328 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-18T12:54:06.337 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2024-06-18T12:54:06.342 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-18T12:54:06.348 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2024-06-18T12:54:06.363 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-18T12:54:06.366 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: ntpd: command not found 2024-06-18T12:54:06.377 INFO:teuthology.orchestra.run.smithi179.stdout:506 Cannot talk to daemon 2024-06-18T12:54:06.377 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-18T12:54:06.391 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-18T12:54:06.405 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-18T12:54:06.439 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-06-18T12:54:06.443 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-18T12:54:06.443 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-06-18T12:54:06.443 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-18T12:54:06.443 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-18T12:54:06.443 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-18T12:54:06.443 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-18T12:54:06.467 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-06-18T12:54:06.471 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-18T12:54:06.471 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-06-18T12:54:06.471 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-18T12:54:06.471 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-18T12:54:06.471 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-18T12:54:06.471 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-18T12:54:06.472 INFO:teuthology.run_tasks:Running task install... 2024-06-18T12:54:06.479 DEBUG:teuthology.task.install:project ceph 2024-06-18T12:54:06.479 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3781e3a592387d16e8020f5db99cdee4198f6d00'}} 2024-06-18T12:54:06.479 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '3781e3a592387d16e8020f5db99cdee4198f6d00'} 2024-06-18T12:54:06.480 INFO:teuthology.task.install:Using flavor: default 2024-06-18T12:54:06.485 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-18T12:54:06.485 INFO:teuthology.task.install:extra packages: [] 2024-06-18T12:54:06.485 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3781e3a592387d16e8020f5db99cdee4198f6d00', 'tag': None, 'wait_for_package': False} 2024-06-18T12:54:06.486 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3781e3a592387d16e8020f5db99cdee4198f6d00 2024-06-18T12:54:06.488 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3781e3a592387d16e8020f5db99cdee4198f6d00', 'tag': None, 'wait_for_package': False} 2024-06-18T12:54:06.488 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3781e3a592387d16e8020f5db99cdee4198f6d00 2024-06-18T12:54:06.634 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/3781e3a592387d16e8020f5db99cdee4198f6d00/centos/9/flavors/default/ 2024-06-18T12:54:06.634 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3124.g3781e3a5 2024-06-18T12:54:06.637 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/3781e3a592387d16e8020f5db99cdee4198f6d00/centos/9/flavors/default/ 2024-06-18T12:54:06.637 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3124.g3781e3a5 2024-06-18T12:54:06.747 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/3781e3a592387d16e8020f5db99cdee4198f6d00/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/3781e3a592387d16e8020f5db99cdee4198f6d00/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/3781e3a592387d16e8020f5db99cdee4198f6d00/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-18T12:54:06.747 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-18T12:54:06.747 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-18T12:54:06.773 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-06-18T12:54:06.773 DEBUG:teuthology.orchestra.run.smithi179:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3781e3a592387d16e8020f5db99cdee4198f6d00/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-18T12:54:06.807 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/3781e3a592387d16e8020f5db99cdee4198f6d00/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/3781e3a592387d16e8020f5db99cdee4198f6d00/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/3781e3a592387d16e8020f5db99cdee4198f6d00/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-18T12:54:06.807 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T12:54:06.807 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-18T12:54:06.833 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-06-18T12:54:06.833 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3781e3a592387d16e8020f5db99cdee4198f6d00/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-18T12:54:06.838 DEBUG:teuthology.orchestra.run.smithi179:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-18T12:54:06.899 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-18T12:54:06.913 DEBUG:teuthology.orchestra.run.smithi179:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-18T12:54:06.975 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-18T12:54:06.977 INFO:teuthology.orchestra.run.smithi179.stdout:check_obsoletes = 1 2024-06-18T12:54:06.978 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum clean all 2024-06-18T12:54:07.039 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2024-06-18T12:54:07.041 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2024-06-18T12:54:07.206 INFO:teuthology.orchestra.run.smithi179.stdout:45 files removed 2024-06-18T12:54:07.233 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-radosgw 2024-06-18T12:54:07.269 INFO:teuthology.orchestra.run.smithi167.stdout:45 files removed 2024-06-18T12:54:07.295 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw 2024-06-18T12:54:07.862 INFO:teuthology.orchestra.run.smithi179.stdout:ceph packages for x86_64 235 kB/s | 83 kB 00:00 2024-06-18T12:54:08.050 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 175 kB/s | 83 kB 00:00 2024-06-18T12:54:08.279 INFO:teuthology.orchestra.run.smithi179.stdout:ceph noarch packages 32 kB/s | 11 kB 00:00 2024-06-18T12:54:08.433 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-06-18T12:54:08.671 INFO:teuthology.orchestra.run.smithi179.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-06-18T12:54:08.818 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-06-18T12:54:09.593 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-06-18T12:54:11.619 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-06-18T12:54:11.672 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.1 MB 00:02 2024-06-18T12:54:18.011 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 2.3 MB/s | 6.3 MB 00:02 2024-06-18T12:54:19.687 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 17 kB 00:00 2024-06-18T12:54:20.906 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - AppStream 2.4 MB/s | 20 MB 00:08 2024-06-18T12:54:21.125 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-06-18T12:54:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.3 MB 00:01 2024-06-18T12:54:26.376 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-06-18T12:54:27.624 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 17 kB 00:00 2024-06-18T12:54:28.767 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:54:28.781 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:54:28.781 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-06-18T12:54:28.781 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:54:28.781 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:54:28.782 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 10 M 2024-06-18T12:54:28.782 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2024-06-18T12:54:28.782 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 3.4 M 2024-06-18T12:54:28.782 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 3.1 M 2024-06-18T12:54:28.782 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-06-18T12:54:28.783 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-18T12:54:28.783 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 5.4 M 2024-06-18T12:54:28.783 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 21 M 2024-06-18T12:54:28.783 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 25 k 2024-06-18T12:54:28.783 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-18T12:54:28.784 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-18T12:54:28.784 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-18T12:54:28.784 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 748 k 2024-06-18T12:54:28.784 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-18T12:54:28.784 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-18T12:54:28.784 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-18T12:54:28.785 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 504 k 2024-06-18T12:54:28.785 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-18T12:54:28.785 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 5.3 M 2024-06-18T12:54:28.785 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-18T12:54:28.785 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-18T12:54:28.786 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-18T12:54:28.786 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 45 k 2024-06-18T12:54:28.786 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 135 k 2024-06-18T12:54:28.786 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 162 k 2024-06-18T12:54:28.786 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-18T12:54:28.787 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 321 k 2024-06-18T12:54:28.787 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 299 k 2024-06-18T12:54:28.787 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 99 k 2024-06-18T12:54:28.787 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-18T12:54:28.787 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-18T12:54:28.787 INFO:teuthology.orchestra.run.smithi167.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-18T12:54:28.788 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-18T12:54:28.788 INFO:teuthology.orchestra.run.smithi167.stdout:Installing weak dependencies: 2024-06-18T12:54:28.788 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-18T12:54:28.788 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:54:28.788 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:54:28.788 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:54:28.789 INFO:teuthology.orchestra.run.smithi167.stdout:Install 30 Packages 2024-06-18T12:54:28.789 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 2 Packages 2024-06-18T12:54:28.789 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:54:28.790 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 60 M 2024-06-18T12:54:28.790 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:54:28.812 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-06-18T12:54:29.802 INFO:teuthology.orchestra.run.smithi167.stdout:(1/32): ceph-base-19.0.0-3124.g3781e3a5.el9.x86 6.8 MB/s | 5.4 MB 00:00 2024-06-18T12:54:29.869 INFO:teuthology.orchestra.run.smithi167.stdout:(2/32): ceph-selinux-19.0.0-3124.g3781e3a5.el9. 378 kB/s | 25 kB 00:00 2024-06-18T12:54:30.003 INFO:teuthology.orchestra.run.smithi167.stdout:(3/32): libcephfs2-19.0.0-3124.g3781e3a5.el9.x8 5.5 MB/s | 748 kB 00:00 2024-06-18T12:54:30.111 INFO:teuthology.orchestra.run.smithi167.stdout:(4/32): libradosstriper1-19.0.0-3124.g3781e3a5. 4.6 MB/s | 504 kB 00:00 2024-06-18T12:54:30.620 INFO:teuthology.orchestra.run.smithi167.stdout:(5/32): librgw2-19.0.0-3124.g3781e3a5.el9.x86_6 10 MB/s | 5.3 MB 00:00 2024-06-18T12:54:30.712 INFO:teuthology.orchestra.run.smithi167.stdout:(6/32): python3-ceph-argparse-19.0.0-3124.g3781 492 kB/s | 45 kB 00:00 2024-06-18T12:54:30.804 INFO:teuthology.orchestra.run.smithi167.stdout:(7/32): python3-ceph-common-19.0.0-3124.g3781e3 1.4 MB/s | 135 kB 00:00 2024-06-18T12:54:30.971 INFO:teuthology.orchestra.run.smithi167.stdout:(8/32): ceph-radosgw-19.0.0-3124.g3781e3a5.el9. 5.2 MB/s | 10 MB 00:01 2024-06-18T12:54:30.997 INFO:teuthology.orchestra.run.smithi167.stdout:(9/32): python3-cephfs-19.0.0-3124.g3781e3a5.el 842 kB/s | 162 kB 00:00 2024-06-18T12:54:31.055 INFO:teuthology.orchestra.run.smithi167.stdout:(10/32): python3-rados-19.0.0-3124.g3781e3a5.el 3.8 MB/s | 321 kB 00:00 2024-06-18T12:54:31.080 INFO:teuthology.orchestra.run.smithi167.stdout:(11/32): python3-rbd-19.0.0-3124.g3781e3a5.el9. 3.5 MB/s | 299 kB 00:00 2024-06-18T12:54:31.297 INFO:teuthology.orchestra.run.smithi167.stdout:(12/32): ceph-common-19.0.0-3124.g3781e3a5.el9. 9.2 MB/s | 21 MB 00:02 2024-06-18T12:54:31.322 INFO:teuthology.orchestra.run.smithi167.stdout:(13/32): python3-rgw-19.0.0-3124.g3781e3a5.el9. 371 kB/s | 99 kB 00:00 2024-06-18T12:54:31.456 INFO:teuthology.orchestra.run.smithi167.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-06-18T12:54:31.473 INFO:teuthology.orchestra.run.smithi167.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 272 kB/s | 107 kB 00:00 2024-06-18T12:54:31.498 INFO:teuthology.orchestra.run.smithi167.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 798 kB/s | 160 kB 00:00 2024-06-18T12:54:31.649 INFO:teuthology.orchestra.run.smithi167.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 281 kB/s | 42 kB 00:00 2024-06-18T12:54:31.707 INFO:teuthology.orchestra.run.smithi167.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2024-06-18T12:54:31.741 INFO:teuthology.orchestra.run.smithi167.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.1 MB/s | 292 kB 00:00 2024-06-18T12:54:31.766 INFO:teuthology.orchestra.run.smithi167.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 611 kB/s | 36 kB 00:00 2024-06-18T12:54:31.791 INFO:teuthology.orchestra.run.smithi167.stdout:(21/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-06-18T12:54:31.817 INFO:teuthology.orchestra.run.smithi167.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-06-18T12:54:31.942 INFO:teuthology.orchestra.run.smithi167.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.7 MB/s | 309 kB 00:00 2024-06-18T12:54:31.967 INFO:teuthology.orchestra.run.smithi167.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 262 kB/s | 26 kB 00:00 2024-06-18T12:54:31.993 INFO:teuthology.orchestra.run.smithi167.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 966 kB/s | 49 kB 00:00 2024-06-18T12:54:32.018 INFO:teuthology.orchestra.run.smithi167.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-06-18T12:54:32.060 INFO:teuthology.orchestra.run.smithi167.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-06-18T12:54:32.110 INFO:teuthology.orchestra.run.smithi167.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-06-18T12:54:32.211 INFO:teuthology.orchestra.run.smithi167.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-06-18T12:54:32.294 INFO:teuthology.orchestra.run.smithi167.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-06-18T12:54:32.536 INFO:teuthology.orchestra.run.smithi167.stdout:(31/32): librados2-19.0.0-3124.g3781e3a5.el9.x8 8.0 MB/s | 3.4 MB 00:00 2024-06-18T12:54:32.603 INFO:teuthology.orchestra.run.smithi167.stdout:(32/32): librbd1-19.0.0-3124.g3781e3a5.el9.x86_ 8.0 MB/s | 3.1 MB 00:00 2024-06-18T12:54:32.607 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:54:32.607 INFO:teuthology.orchestra.run.smithi167.stdout:Total 16 MB/s | 60 MB 00:03 2024-06-18T12:54:32.706 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:54:32.741 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:54:32.741 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:54:33.306 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:54:33.307 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:54:34.085 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-06-18T12:54:34.739 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:54:34.823 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-06-18T12:54:34.858 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-06-18T12:54:34.883 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-06-18T12:54:34.914 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-06-18T12:54:34.934 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-06-18T12:54:35.124 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-06-18T12:54:35.181 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-06-18T12:54:35.577 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-06-18T12:54:35.627 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:19.0.0-3124.g3781e3a5.el9.x86_64 7/34 2024-06-18T12:54:35.710 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:19.0.0-3124.g3781e3a5.el9.x86_64 7/34 2024-06-18T12:54:35.798 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:19.0.0-3124.g3781e3a5.el9.x86_64 8/34 2024-06-18T12:54:35.837 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:19.0.0-3124.g3781e3a5.el9.x86_64 9/34 2024-06-18T12:54:35.900 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:19.0.0-3124.g3781e3a5.el9.x86_64 9/34 2024-06-18T12:54:35.928 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-06-18T12:54:35.964 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:19.0.0-3124.g3781e3a5.el9. 11/34 2024-06-18T12:54:36.032 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:19.0.0-3124.g3781e3a5.el9.x86_64 12/34 2024-06-18T12:54:36.056 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:19.0.0-3124.g3781e3a5.el9.x86_6 13/34 2024-06-18T12:54:36.120 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3124.g3781e3a5.el9.x86_6 13/34 2024-06-18T12:54:36.163 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-06-18T12:54:36.281 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-06-18T12:54:36.315 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-06-18T12:54:36.372 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-06-18T12:54:36.404 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-06-18T12:54:36.433 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-06-18T12:54:36.466 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-06-18T12:54:36.502 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:54:36.506 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:54:36.506 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-06-18T12:54:36.506 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:54:36.506 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:54:36.506 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 10 M 2024-06-18T12:54:36.506 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrading: 2024-06-18T12:54:36.507 INFO:teuthology.orchestra.run.smithi179.stdout: librados2 x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 3.4 M 2024-06-18T12:54:36.507 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1 x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 3.1 M 2024-06-18T12:54:36.507 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-06-18T12:54:36.507 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-18T12:54:36.507 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 5.4 M 2024-06-18T12:54:36.507 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 21 M 2024-06-18T12:54:36.507 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 25 k 2024-06-18T12:54:36.507 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-18T12:54:36.507 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-18T12:54:36.507 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-18T12:54:36.507 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2 x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 748 k 2024-06-18T12:54:36.507 INFO:teuthology.orchestra.run.smithi179.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-18T12:54:36.507 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-18T12:54:36.508 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-18T12:54:36.508 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1 x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 504 k 2024-06-18T12:54:36.508 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-18T12:54:36.508 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2 x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 5.3 M 2024-06-18T12:54:36.508 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-18T12:54:36.508 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-18T12:54:36.508 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-18T12:54:36.508 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 45 k 2024-06-18T12:54:36.508 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 135 k 2024-06-18T12:54:36.508 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 162 k 2024-06-18T12:54:36.508 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-18T12:54:36.508 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 321 k 2024-06-18T12:54:36.508 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 299 k 2024-06-18T12:54:36.509 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 99 k 2024-06-18T12:54:36.509 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-18T12:54:36.509 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-18T12:54:36.509 INFO:teuthology.orchestra.run.smithi179.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-18T12:54:36.509 INFO:teuthology.orchestra.run.smithi179.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-18T12:54:36.509 INFO:teuthology.orchestra.run.smithi179.stdout:Installing weak dependencies: 2024-06-18T12:54:36.509 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-18T12:54:36.509 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:54:36.509 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:54:36.509 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:54:36.509 INFO:teuthology.orchestra.run.smithi179.stdout:Install 30 Packages 2024-06-18T12:54:36.509 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrade 2 Packages 2024-06-18T12:54:36.509 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:54:36.510 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 60 M 2024-06-18T12:54:36.510 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:54:36.779 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-06-18T12:54:36.821 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:19.0.0-3124.g3781e3a5.el9.x86_64 22/34 2024-06-18T12:54:36.910 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:19.0.0-3124.g3781e3a5.el9.x86_64 22/34 2024-06-18T12:54:36.936 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 23/34 2024-06-18T12:54:36.978 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-06-18T12:54:37.449 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-06-18T12:54:37.482 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:19.0.0-3124.g3781e3a5.el9.x86_64 26/34 2024-06-18T12:54:37.553 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:19.0.0-3124.g3781e3a5.el9.x86_64 26/34 2024-06-18T12:54:37.595 INFO:teuthology.orchestra.run.smithi179.stdout:(1/32): ceph-radosgw-19.0.0-3124.g3781e3a5.el9. 13 MB/s | 10 MB 00:00 2024-06-18T12:54:37.623 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 27/34 2024-06-18T12:54:37.645 INFO:teuthology.orchestra.run.smithi179.stdout:(2/32): ceph-selinux-19.0.0-3124.g3781e3a5.el9. 504 kB/s | 25 kB 00:00 2024-06-18T12:54:37.651 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:19.0.0-3124.g3781e3a5.el9.x8 28/34 2024-06-18T12:54:37.729 INFO:teuthology.orchestra.run.smithi179.stdout:(3/32): libcephfs2-19.0.0-3124.g3781e3a5.el9.x8 8.8 MB/s | 748 kB 00:00 2024-06-18T12:54:37.788 INFO:teuthology.orchestra.run.smithi179.stdout:(4/32): libradosstriper1-19.0.0-3124.g3781e3a5. 8.4 MB/s | 504 kB 00:00 2024-06-18T12:54:38.130 INFO:teuthology.orchestra.run.smithi179.stdout:(5/32): librgw2-19.0.0-3124.g3781e3a5.el9.x86_6 16 MB/s | 5.3 MB 00:00 2024-06-18T12:54:38.188 INFO:teuthology.orchestra.run.smithi179.stdout:(6/32): python3-ceph-argparse-19.0.0-3124.g3781 773 kB/s | 45 kB 00:00 2024-06-18T12:54:38.263 INFO:teuthology.orchestra.run.smithi179.stdout:(7/32): ceph-base-19.0.0-3124.g3781e3a5.el9.x86 3.6 MB/s | 5.4 MB 00:01 2024-06-18T12:54:38.289 INFO:teuthology.orchestra.run.smithi179.stdout:(8/32): python3-ceph-common-19.0.0-3124.g3781e3 1.3 MB/s | 135 kB 00:00 2024-06-18T12:54:38.332 INFO:teuthology.orchestra.run.smithi179.stdout:(9/32): python3-cephfs-19.0.0-3124.g3781e3a5.el 2.4 MB/s | 162 kB 00:00 2024-06-18T12:54:38.356 INFO:teuthology.orchestra.run.smithi179.stdout:(10/32): python3-rados-19.0.0-3124.g3781e3a5.el 4.7 MB/s | 321 kB 00:00 2024-06-18T12:54:38.398 INFO:teuthology.orchestra.run.smithi179.stdout:(11/32): python3-rbd-19.0.0-3124.g3781e3a5.el9. 4.3 MB/s | 299 kB 00:00 2024-06-18T12:54:38.424 INFO:teuthology.orchestra.run.smithi179.stdout:(12/32): python3-rgw-19.0.0-3124.g3781e3a5.el9. 1.5 MB/s | 99 kB 00:00 2024-06-18T12:54:38.916 INFO:teuthology.orchestra.run.smithi179.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 207 kB/s | 107 kB 00:00 2024-06-18T12:54:38.966 INFO:teuthology.orchestra.run.smithi179.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 295 kB/s | 160 kB 00:00 2024-06-18T12:54:39.033 INFO:teuthology.orchestra.run.smithi179.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-06-18T12:54:39.292 INFO:teuthology.orchestra.run.smithi179.stdout:(16/32): ceph-common-19.0.0-3124.g3781e3a5.el9. 8.4 MB/s | 21 MB 00:02 2024-06-18T12:54:39.325 INFO:teuthology.orchestra.run.smithi179.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 1.8 MB/s | 662 kB 00:00 2024-06-18T12:54:39.401 INFO:teuthology.orchestra.run.smithi179.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 796 kB/s | 292 kB 00:00 2024-06-18T12:54:39.426 INFO:teuthology.orchestra.run.smithi179.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 317 kB/s | 42 kB 00:00 2024-06-18T12:54:39.460 INFO:teuthology.orchestra.run.smithi179.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-06-18T12:54:39.510 INFO:teuthology.orchestra.run.smithi179.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 328 kB/s | 36 kB 00:00 2024-06-18T12:54:39.536 INFO:teuthology.orchestra.run.smithi179.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-06-18T12:54:39.636 INFO:teuthology.orchestra.run.smithi179.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.7 MB/s | 309 kB 00:00 2024-06-18T12:54:39.661 INFO:teuthology.orchestra.run.smithi179.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-06-18T12:54:39.686 INFO:teuthology.orchestra.run.smithi179.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 206 kB/s | 26 kB 00:00 2024-06-18T12:54:39.712 INFO:teuthology.orchestra.run.smithi179.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-06-18T12:54:39.754 INFO:teuthology.orchestra.run.smithi179.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-06-18T12:54:39.813 INFO:teuthology.orchestra.run.smithi179.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-06-18T12:54:39.888 INFO:teuthology.orchestra.run.smithi179.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-06-18T12:54:39.955 INFO:teuthology.orchestra.run.smithi179.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-06-18T12:54:39.961 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.g3781e3a5.el9.x86_64 29/34 2024-06-18T12:54:39.998 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:19.0.0-3124.g3781e3a5.el9.x86_64 29/34 2024-06-18T12:54:40.188 INFO:teuthology.orchestra.run.smithi179.stdout:(31/32): librbd1-19.0.0-3124.g3781e3a5.el9.x86_ 10 MB/s | 3.1 MB 00:00 2024-06-18T12:54:40.422 INFO:teuthology.orchestra.run.smithi179.stdout:(32/32): librados2-19.0.0-3124.g3781e3a5.el9.x8 5.6 MB/s | 3.4 MB 00:00 2024-06-18T12:54:40.428 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:54:40.428 INFO:teuthology.orchestra.run.smithi179.stdout:Total 15 MB/s | 60 MB 00:03 2024-06-18T12:54:40.526 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:54:40.556 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.g3781e3a5.el9.x86_64 29/34 2024-06-18T12:54:40.561 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:54:40.561 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:54:40.594 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:19.0.0-3124.g3781e3a5.el9.x86_64 30/34 2024-06-18T12:54:40.683 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:19.0.0-3124.g3781e3a5.el9.x86_64 30/34 2024-06-18T12:54:40.683 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-18T12:54:40.683 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-18T12:54:40.683 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:54:40.712 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:19.0.0-3124.g3781e3a5.el9.x86_64 31/34 2024-06-18T12:54:41.108 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:54:41.109 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:54:42.523 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:54:42.644 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-06-18T12:54:42.686 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-06-18T12:54:42.720 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-06-18T12:54:42.743 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-06-18T12:54:42.767 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-06-18T12:54:42.962 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-06-18T12:54:43.017 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-06-18T12:54:43.419 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-06-18T12:54:43.481 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librados2-2:19.0.0-3124.g3781e3a5.el9.x86_64 7/34 2024-06-18T12:54:43.573 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:19.0.0-3124.g3781e3a5.el9.x86_64 7/34 2024-06-18T12:54:43.662 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rados-2:19.0.0-3124.g3781e3a5.el9.x86_64 8/34 2024-06-18T12:54:43.691 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs2-2:19.0.0-3124.g3781e3a5.el9.x86_64 9/34 2024-06-18T12:54:43.745 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs2-2:19.0.0-3124.g3781e3a5.el9.x86_64 9/34 2024-06-18T12:54:43.774 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-06-18T12:54:43.818 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-argparse-2:19.0.0-3124.g3781e3a5.el9. 11/34 2024-06-18T12:54:43.888 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cephfs-2:19.0.0-3124.g3781e3a5.el9.x86_64 12/34 2024-06-18T12:54:43.910 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libradosstriper1-2:19.0.0-3124.g3781e3a5.el9.x86_6 13/34 2024-06-18T12:54:43.966 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3124.g3781e3a5.el9.x86_6 13/34 2024-06-18T12:54:43.993 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-06-18T12:54:44.105 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-06-18T12:54:44.136 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-06-18T12:54:44.176 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-06-18T12:54:44.209 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-06-18T12:54:44.238 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-06-18T12:54:44.271 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-06-18T12:54:44.583 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-06-18T12:54:44.635 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librbd1-2:19.0.0-3124.g3781e3a5.el9.x86_64 22/34 2024-06-18T12:54:44.724 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:19.0.0-3124.g3781e3a5.el9.x86_64 22/34 2024-06-18T12:54:44.757 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 23/34 2024-06-18T12:54:44.793 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-06-18T12:54:45.264 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-06-18T12:54:45.288 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librgw2-2:19.0.0-3124.g3781e3a5.el9.x86_64 26/34 2024-06-18T12:54:45.358 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librgw2-2:19.0.0-3124.g3781e3a5.el9.x86_64 26/34 2024-06-18T12:54:45.427 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 27/34 2024-06-18T12:54:45.458 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-common-2:19.0.0-3124.g3781e3a5.el9.x8 28/34 2024-06-18T12:54:47.694 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.g3781e3a5.el9.x86_64 29/34 2024-06-18T12:54:47.731 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-common-2:19.0.0-3124.g3781e3a5.el9.x86_64 29/34 2024-06-18T12:54:48.302 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.g3781e3a5.el9.x86_64 29/34 2024-06-18T12:54:48.344 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-base-2:19.0.0-3124.g3781e3a5.el9.x86_64 30/34 2024-06-18T12:54:48.436 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-base-2:19.0.0-3124.g3781e3a5.el9.x86_64 30/34 2024-06-18T12:54:48.436 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-18T12:54:48.436 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-18T12:54:48.436 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:54:48.470 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-selinux-2:19.0.0-3124.g3781e3a5.el9.x86_64 31/34 2024-06-18T12:54:50.568 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3124.g3781e3a5.el9.x86_64 31/34 2024-06-18T12:54:50.568 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2024-06-18T12:54:50.568 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2024-06-18T12:54:50.568 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2024-06-18T12:54:50.568 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2024-06-18T12:54:50.568 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2024-06-18T12:54:50.568 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /root 2024-06-18T12:54:50.568 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2024-06-18T12:54:50.568 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:54:51.551 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 32/34 2024-06-18T12:54:51.576 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 32/34 2024-06-18T12:54:51.577 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:54:51.577 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-18T12:54:51.577 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-18T12:54:51.577 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-18T12:54:51.577 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:54:51.579 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-06-18T12:54:51.653 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-06-18T12:54:51.653 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-06-18T12:54:52.925 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-06-18T12:54:52.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/34 2024-06-18T12:54:52.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:19.0.0-3124.g3781e3a5.el9.x86_64 2/34 2024-06-18T12:54:52.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 3/34 2024-06-18T12:54:52.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:19.0.0-3124.g3781e3a5.el9.x86_64 4/34 2024-06-18T12:54:52.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:19.0.0-3124.g3781e3a5.el9.x86_64 5/34 2024-06-18T12:54:52.925 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:19.0.0-3124.g3781e3a5.el9.x86_6 6/34 2024-06-18T12:54:52.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:19.0.0-3124.g3781e3a5.el9.x86_64 7/34 2024-06-18T12:54:52.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3124.g3781e3a5.el9. 8/34 2024-06-18T12:54:52.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:19.0.0-3124.g3781e3a5.el9.x8 9/34 2024-06-18T12:54:52.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:19.0.0-3124.g3781e3a5.el9.x86_64 10/34 2024-06-18T12:54:52.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:19.0.0-3124.g3781e3a5.el9.x86_64 11/34 2024-06-18T12:54:52.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 12/34 2024-06-18T12:54:52.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 13/34 2024-06-18T12:54:52.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-06-18T12:54:52.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-06-18T12:54:52.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-06-18T12:54:52.926 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-06-18T12:54:52.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-06-18T12:54:52.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-06-18T12:54:52.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-06-18T12:54:52.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-06-18T12:54:52.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-06-18T12:54:52.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-06-18T12:54:52.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-06-18T12:54:52.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-06-18T12:54:52.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-06-18T12:54:52.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-06-18T12:54:52.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-06-18T12:54:52.927 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-06-18T12:54:52.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-06-18T12:54:52.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:19.0.0-3124.g3781e3a5.el9.x86_64 31/34 2024-06-18T12:54:52.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-06-18T12:54:52.928 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:19.0.0-3124.g3781e3a5.el9.x86_64 33/34 2024-06-18T12:54:53.712 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-06-18T12:54:53.713 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:54:53.713 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2024-06-18T12:54:53.713 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.713 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.713 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:54:53.713 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-18T12:54:53.713 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.713 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.713 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.713 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.714 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-18T12:54:53.714 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-18T12:54:53.714 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-18T12:54:53.714 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.714 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-18T12:54:53.714 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-18T12:54:53.714 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-18T12:54:53.714 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.714 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-18T12:54:53.714 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.714 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-18T12:54:53.714 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-18T12:54:53.714 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-18T12:54:53.715 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.715 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.715 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.715 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-18T12:54:53.715 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.715 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.715 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:54:53.715 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-18T12:54:53.715 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-18T12:54:53.715 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-18T12:54:53.715 INFO:teuthology.orchestra.run.smithi167.stdout: re2-1:20211101-3.el9.x86_64 2024-06-18T12:54:53.715 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-18T12:54:53.716 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:54:53.716 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:54:54.072 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-test 2024-06-18T12:54:54.645 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:28 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:54:54.768 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:54:54.769 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:54:54.769 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-06-18T12:54:54.769 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:54:54.769 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:54:54.770 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 48 M 2024-06-18T12:54:54.770 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-06-18T12:54:54.770 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 163 k 2024-06-18T12:54:54.770 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-06-18T12:54:54.770 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-18T12:54:54.770 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:54:54.771 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:54:54.771 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:54:54.771 INFO:teuthology.orchestra.run.smithi167.stdout:Install 4 Packages 2024-06-18T12:54:54.771 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:54:54.772 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 49 M 2024-06-18T12:54:54.772 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 207 M 2024-06-18T12:54:54.772 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:54:55.558 INFO:teuthology.orchestra.run.smithi167.stdout:(1/4): libcephsqlite-19.0.0-3124.g3781e3a5.el9. 701 kB/s | 163 kB 00:00 2024-06-18T12:54:55.583 INFO:teuthology.orchestra.run.smithi167.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-06-18T12:54:55.700 INFO:teuthology.orchestra.run.smithi167.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 449 kB/s | 64 kB 00:00 2024-06-18T12:54:57.850 INFO:teuthology.orchestra.run.smithi167.stdout:(4/4): ceph-test-19.0.0-3124.g3781e3a5.el9.x86_ 19 MB/s | 48 MB 00:02 2024-06-18T12:54:57.852 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:54:57.853 INFO:teuthology.orchestra.run.smithi167.stdout:Total 16 MB/s | 49 MB 00:03 2024-06-18T12:54:57.880 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:54:57.901 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:54:57.901 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:54:58.163 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3124.g3781e3a5.el9.x86_64 31/34 2024-06-18T12:54:58.163 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /sys 2024-06-18T12:54:58.163 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /proc 2024-06-18T12:54:58.163 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /mnt 2024-06-18T12:54:58.163 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /var/tmp 2024-06-18T12:54:58.164 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /home 2024-06-18T12:54:58.164 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /root 2024-06-18T12:54:58.164 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /tmp 2024-06-18T12:54:58.164 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:54:58.269 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:54:58.269 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:54:58.783 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:54:58.851 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-06-18T12:54:58.891 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-06-18T12:54:58.919 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:19.0.0-3124.g3781e3a5.el9.x86_64 3/4 2024-06-18T12:54:59.137 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-radosgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 32/34 2024-06-18T12:54:59.162 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 32/34 2024-06-18T12:54:59.162 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:54:59.162 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-18T12:54:59.162 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-18T12:54:59.162 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-18T12:54:59.163 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:54:59.165 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-06-18T12:54:59.257 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-06-18T12:54:59.257 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-06-18T12:55:00.389 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-06-18T12:55:00.389 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-base-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/34 2024-06-18T12:55:00.389 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-common-2:19.0.0-3124.g3781e3a5.el9.x86_64 2/34 2024-06-18T12:55:00.389 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-radosgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 3/34 2024-06-18T12:55:00.389 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-selinux-2:19.0.0-3124.g3781e3a5.el9.x86_64 4/34 2024-06-18T12:55:00.389 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs2-2:19.0.0-3124.g3781e3a5.el9.x86_64 5/34 2024-06-18T12:55:00.390 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libradosstriper1-2:19.0.0-3124.g3781e3a5.el9.x86_6 6/34 2024-06-18T12:55:00.390 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librgw2-2:19.0.0-3124.g3781e3a5.el9.x86_64 7/34 2024-06-18T12:55:00.390 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3124.g3781e3a5.el9. 8/34 2024-06-18T12:55:00.390 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-common-2:19.0.0-3124.g3781e3a5.el9.x8 9/34 2024-06-18T12:55:00.390 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cephfs-2:19.0.0-3124.g3781e3a5.el9.x86_64 10/34 2024-06-18T12:55:00.390 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rados-2:19.0.0-3124.g3781e3a5.el9.x86_64 11/34 2024-06-18T12:55:00.390 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 12/34 2024-06-18T12:55:00.390 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 13/34 2024-06-18T12:55:00.390 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-06-18T12:55:00.390 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-06-18T12:55:00.390 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-06-18T12:55:00.390 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-06-18T12:55:00.391 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-06-18T12:55:00.391 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-06-18T12:55:00.391 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-06-18T12:55:00.391 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-06-18T12:55:00.391 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-06-18T12:55:00.391 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-06-18T12:55:00.391 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-06-18T12:55:00.391 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-06-18T12:55:00.391 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-06-18T12:55:00.391 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-06-18T12:55:00.391 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-06-18T12:55:00.391 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-06-18T12:55:00.391 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-06-18T12:55:00.392 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:19.0.0-3124.g3781e3a5.el9.x86_64 31/34 2024-06-18T12:55:00.392 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-06-18T12:55:00.392 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:19.0.0-3124.g3781e3a5.el9.x86_64 33/34 2024-06-18T12:55:01.189 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-06-18T12:55:01.189 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:01.189 INFO:teuthology.orchestra.run.smithi179.stdout:Upgraded: 2024-06-18T12:55:01.189 INFO:teuthology.orchestra.run.smithi179.stdout: librados2-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.189 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.189 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:55:01.189 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-18T12:55:01.190 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.190 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.190 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.190 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.190 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-18T12:55:01.190 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-18T12:55:01.190 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-18T12:55:01.190 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.190 INFO:teuthology.orchestra.run.smithi179.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-18T12:55:01.190 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-18T12:55:01.190 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-18T12:55:01.190 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.190 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-18T12:55:01.191 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.191 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-18T12:55:01.191 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-18T12:55:01.191 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-18T12:55:01.191 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.191 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.191 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.191 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-18T12:55:01.191 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.191 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.191 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:01.192 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-18T12:55:01.192 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-18T12:55:01.192 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-18T12:55:01.192 INFO:teuthology.orchestra.run.smithi179.stdout: re2-1:20211101-3.el9.x86_64 2024-06-18T12:55:01.192 INFO:teuthology.orchestra.run.smithi179.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-18T12:55:01.192 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:01.192 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:55:01.558 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-test 2024-06-18T12:55:02.136 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:28 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:55:02.260 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:55:02.260 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:02.261 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-06-18T12:55:02.261 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:02.261 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:55:02.261 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 48 M 2024-06-18T12:55:02.261 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-06-18T12:55:02.261 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 163 k 2024-06-18T12:55:02.261 INFO:teuthology.orchestra.run.smithi179.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-06-18T12:55:02.261 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-18T12:55:02.261 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:02.261 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:55:02.261 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:02.261 INFO:teuthology.orchestra.run.smithi179.stdout:Install 4 Packages 2024-06-18T12:55:02.261 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:02.262 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 49 M 2024-06-18T12:55:02.262 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 207 M 2024-06-18T12:55:02.262 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:55:02.607 INFO:teuthology.orchestra.run.smithi179.stdout:(1/4): libcephsqlite-19.0.0-3124.g3781e3a5.el9. 719 kB/s | 163 kB 00:00 2024-06-18T12:55:03.274 INFO:teuthology.orchestra.run.smithi179.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 340 kB/s | 304 kB 00:00 2024-06-18T12:55:03.324 INFO:teuthology.orchestra.run.smithi179.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 92 kB/s | 64 kB 00:00 2024-06-18T12:55:03.485 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3124.g3781e3a5.el9.x86_64 3/4 2024-06-18T12:55:03.558 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:19.0.0-3124.g3781e3a5.el9.x86_64 4/4 2024-06-18T12:55:04.899 INFO:teuthology.orchestra.run.smithi179.stdout:(4/4): ceph-test-19.0.0-3124.g3781e3a5.el9.x86_ 19 MB/s | 48 MB 00:02 2024-06-18T12:55:04.902 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:55:04.902 INFO:teuthology.orchestra.run.smithi179.stdout:Total 19 MB/s | 49 MB 00:02 2024-06-18T12:55:04.929 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:55:04.949 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:55:04.949 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:55:05.316 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:55:05.316 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:55:05.835 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:55:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-test-2:19.0.0-3124.g3781e3a5.el9.x86_64 4/4 2024-06-18T12:55:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/4 2024-06-18T12:55:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:19.0.0-3124.g3781e3a5.el9.x86_64 2/4 2024-06-18T12:55:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-06-18T12:55:05.908 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-06-18T12:55:05.956 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-06-18T12:55:05.984 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephsqlite-2:19.0.0-3124.g3781e3a5.el9.x86_64 3/4 2024-06-18T12:55:06.165 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-06-18T12:55:06.166 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:06.166 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:55:06.166 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:06.166 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:06.166 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-6.el9.x86_64 2024-06-18T12:55:06.166 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-18T12:55:06.166 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:06.166 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:55:06.470 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph 2024-06-18T12:55:07.042 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:41 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:55:07.170 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:55:07.172 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:55:07.173 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-06-18T12:55:07.173 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:55:07.173 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:55:07.173 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 6.6 k 2024-06-18T12:55:07.173 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-06-18T12:55:07.173 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 2.4 M 2024-06-18T12:55:07.173 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 1.5 M 2024-06-18T12:55:07.173 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 249 k 2024-06-18T12:55:07.173 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 4.7 M 2024-06-18T12:55:07.173 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 17 M 2024-06-18T12:55:07.173 INFO:teuthology.orchestra.run.smithi167.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-18T12:55:07.174 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-18T12:55:07.174 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-18T12:55:07.174 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-18T12:55:07.174 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-18T12:55:07.174 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-18T12:55:07.174 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-18T12:55:07.174 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-18T12:55:07.174 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-18T12:55:07.174 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-18T12:55:07.174 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections 2024-06-18T12:55:07.174 INFO:teuthology.orchestra.run.smithi167.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-06-18T12:55:07.174 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-18T12:55:07.175 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-18T12:55:07.175 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-18T12:55:07.175 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-18T12:55:07.175 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-18T12:55:07.175 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-18T12:55:07.175 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-18T12:55:07.175 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-18T12:55:07.175 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-18T12:55:07.175 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-18T12:55:07.175 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-18T12:55:07.175 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-18T12:55:07.175 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-18T12:55:07.175 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-18T12:55:07.175 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-18T12:55:07.176 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-18T12:55:07.176 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-18T12:55:07.176 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-18T12:55:07.176 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-18T12:55:07.176 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:07.176 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:55:07.176 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:55:07.176 INFO:teuthology.orchestra.run.smithi167.stdout:Install 36 Packages 2024-06-18T12:55:07.176 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:07.177 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 30 M 2024-06-18T12:55:07.177 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 107 M 2024-06-18T12:55:07.177 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:55:07.652 INFO:teuthology.orchestra.run.smithi167.stdout:(1/36): ceph-19.0.0-3124.g3781e3a5.el9.x86_64.r 40 kB/s | 6.6 kB 00:00 2024-06-18T12:55:07.877 INFO:teuthology.orchestra.run.smithi167.stdout:(2/36): ceph-mgr-19.0.0-3124.g3781e3a5.el9.x86_ 3.8 MB/s | 1.5 MB 00:00 2024-06-18T12:55:08.136 INFO:teuthology.orchestra.run.smithi167.stdout:(3/36): ceph-mon-19.0.0-3124.g3781e3a5.el9.x86_ 9.7 MB/s | 4.7 MB 00:00 2024-06-18T12:55:08.228 INFO:teuthology.orchestra.run.smithi167.stdout:(4/36): ceph-mgr-modules-core-19.0.0-3124.g3781 2.7 MB/s | 249 kB 00:00 2024-06-18T12:55:08.787 INFO:teuthology.orchestra.run.smithi167.stdout:(5/36): ceph-osd-19.0.0-3124.g3781e3a5.el9.x86_ 18 MB/s | 17 MB 00:00 2024-06-18T12:55:08.828 INFO:teuthology.orchestra.run.smithi167.stdout:(6/36): ceph-mds-19.0.0-3124.g3781e3a5.el9.x86_ 1.8 MB/s | 2.4 MB 00:01 2024-06-18T12:55:09.087 INFO:teuthology.orchestra.run.smithi167.stdout:(7/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 295 kB/s | 253 kB 00:00 2024-06-18T12:55:09.195 INFO:teuthology.orchestra.run.smithi167.stdout:(8/36): python3-ply-3.11-14.el9.noarch.rpm 290 kB/s | 106 kB 00:00 2024-06-18T12:55:09.296 INFO:teuthology.orchestra.run.smithi167.stdout:(9/36): python3-cryptography-36.0.1-4.el9.x86_6 2.4 MB/s | 1.2 MB 00:00 2024-06-18T12:55:09.321 INFO:teuthology.orchestra.run.smithi167.stdout:(10/36): python3-pycparser-2.20-6.el9.noarch.rp 578 kB/s | 135 kB 00:00 2024-06-18T12:55:09.338 INFO:teuthology.orchestra.run.smithi167.stdout:(11/36): python3-requests-2.25.1-8.el9.noarch.r 880 kB/s | 125 kB 00:00 2024-06-18T12:55:09.396 INFO:teuthology.orchestra.run.smithi167.stdout:(12/36): python3-urllib3-1.26.5-5.el9.noarch.rp 2.1 MB/s | 215 kB 00:00 2024-06-18T12:55:09.463 INFO:teuthology.orchestra.run.smithi167.stdout:(13/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-06-18T12:55:09.489 INFO:teuthology.orchestra.run.smithi167.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 378 kB/s | 35 kB 00:00 2024-06-18T12:55:09.514 INFO:teuthology.orchestra.run.smithi167.stdout:(15/36): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-06-18T12:55:09.539 INFO:teuthology.orchestra.run.smithi167.stdout:(16/36): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-06-18T12:55:09.564 INFO:teuthology.orchestra.run.smithi167.stdout:(17/36): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-06-18T12:55:09.657 INFO:teuthology.orchestra.run.smithi167.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 475 kB/s | 43 kB 00:00 2024-06-18T12:55:09.682 INFO:teuthology.orchestra.run.smithi167.stdout:(19/36): luarocks-3.9.2-5.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-06-18T12:55:09.707 INFO:teuthology.orchestra.run.smithi167.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-06-18T12:55:09.732 INFO:teuthology.orchestra.run.smithi167.stdout:(21/36): lua-devel-5.4.4-4.el9.x86_64.rpm 102 kB/s | 22 kB 00:00 2024-06-18T12:55:09.758 INFO:teuthology.orchestra.run.smithi167.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 212 kB/s | 11 kB 00:00 2024-06-18T12:55:09.808 INFO:teuthology.orchestra.run.smithi167.stdout:(23/36): python3-jaraco-collections-3.0.0-8.el9 958 kB/s | 23 kB 00:00 2024-06-18T12:55:09.833 INFO:teuthology.orchestra.run.smithi167.stdout:(24/36): python3-cherrypy-18.6.1-2.el9.noarch.r 2.3 MB/s | 358 kB 00:00 2024-06-18T12:55:09.850 INFO:teuthology.orchestra.run.smithi167.stdout:(25/36): python3-jaraco-classes-3.2.1-5.el9.noa 151 kB/s | 18 kB 00:00 2024-06-18T12:55:09.867 INFO:teuthology.orchestra.run.smithi167.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 345 kB/s | 19 kB 00:00 2024-06-18T12:55:09.884 INFO:teuthology.orchestra.run.smithi167.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 390 kB/s | 20 kB 00:00 2024-06-18T12:55:09.901 INFO:teuthology.orchestra.run.smithi167.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 1.4 MB/s | 46 kB 00:00 2024-06-18T12:55:09.926 INFO:teuthology.orchestra.run.smithi167.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-06-18T12:55:09.943 INFO:teuthology.orchestra.run.smithi167.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 392 kB/s | 16 kB 00:00 2024-06-18T12:55:09.960 INFO:teuthology.orchestra.run.smithi167.stdout:(31/36): python3-tempora-5.0.0-2.el9.noarch.rpm 2.1 MB/s | 36 kB 00:00 2024-06-18T12:55:09.985 INFO:teuthology.orchestra.run.smithi167.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-06-18T12:55:10.011 INFO:teuthology.orchestra.run.smithi167.stdout:(33/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-06-18T12:55:10.036 INFO:teuthology.orchestra.run.smithi167.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-06-18T12:55:10.053 INFO:teuthology.orchestra.run.smithi167.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 477 kB/s | 20 kB 00:00 2024-06-18T12:55:10.086 INFO:teuthology.orchestra.run.smithi167.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-06-18T12:55:10.092 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:55:10.092 INFO:teuthology.orchestra.run.smithi167.stdout:Total 10 MB/s | 30 MB 00:02 2024-06-18T12:55:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:55:10.271 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:55:10.271 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:55:10.571 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3124.g3781e3a5.el9.x86_64 3/4 2024-06-18T12:55:10.649 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-test-2:19.0.0-3124.g3781e3a5.el9.x86_64 4/4 2024-06-18T12:55:10.709 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:55:10.709 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:55:11.966 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:55:12.022 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-06-18T12:55:12.056 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-06-18T12:55:12.083 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-06-18T12:55:12.154 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-06-18T12:55:12.199 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/36 2024-06-18T12:55:12.232 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-06-18T12:55:12.324 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-06-18T12:55:12.403 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-06-18T12:55:12.443 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-06-18T12:55:12.478 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-06-18T12:55:12.508 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-06-18T12:55:12.542 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-06-18T12:55:12.576 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-06-18T12:55:12.608 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-06-18T12:55:12.643 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-06-18T12:55:12.677 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-06-18T12:55:12.715 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-06-18T12:55:12.789 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-06-18T12:55:12.928 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-06-18T12:55:13.015 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-06-18T12:55:13.077 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-06-18T12:55:13.115 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-06-18T12:55:13.173 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-06-18T12:55:13.178 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-test-2:19.0.0-3124.g3781e3a5.el9.x86_64 4/4 2024-06-18T12:55:13.178 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-test-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/4 2024-06-18T12:55:13.178 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephsqlite-2:19.0.0-3124.g3781e3a5.el9.x86_64 2/4 2024-06-18T12:55:13.178 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-06-18T12:55:13.228 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-06-18T12:55:13.416 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-06-18T12:55:13.462 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-06-18T12:55:13.463 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:13.463 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:55:13.463 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:13.463 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:13.463 INFO:teuthology.orchestra.run.smithi179.stdout: socat-1.7.4.1-6.el9.x86_64 2024-06-18T12:55:13.463 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-18T12:55:13.463 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:13.463 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:55:13.469 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-06-18T12:55:13.548 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-06-18T12:55:13.644 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-06-18T12:55:13.694 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-06-18T12:55:13.726 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph 2024-06-18T12:55:13.811 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-06-18T12:55:13.966 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3124.g3781e3a5.el9. 31/36 2024-06-18T12:55:13.996 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:19.0.0-3124.g3781e3a5.el9.x86_64 32/36 2024-06-18T12:55:14.021 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3124.g3781e3a5.el9.x86_64 32/36 2024-06-18T12:55:14.021 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:55:14.022 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-18T12:55:14.022 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-18T12:55:14.022 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-18T12:55:14.022 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:14.325 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:40 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:55:14.461 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:55:14.463 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:14.463 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-06-18T12:55:14.463 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:14.463 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:55:14.463 INFO:teuthology.orchestra.run.smithi179.stdout: ceph x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 6.6 k 2024-06-18T12:55:14.463 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-06-18T12:55:14.464 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 2.4 M 2024-06-18T12:55:14.464 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 1.5 M 2024-06-18T12:55:14.464 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 249 k 2024-06-18T12:55:14.464 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 4.7 M 2024-06-18T12:55:14.464 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 17 M 2024-06-18T12:55:14.464 INFO:teuthology.orchestra.run.smithi179.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-18T12:55:14.464 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-18T12:55:14.464 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-18T12:55:14.464 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-18T12:55:14.464 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-18T12:55:14.464 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-18T12:55:14.465 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-18T12:55:14.465 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-18T12:55:14.465 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-18T12:55:14.465 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-18T12:55:14.465 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections 2024-06-18T12:55:14.465 INFO:teuthology.orchestra.run.smithi179.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-06-18T12:55:14.465 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-18T12:55:14.465 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-18T12:55:14.465 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-18T12:55:14.465 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-18T12:55:14.465 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-18T12:55:14.465 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-18T12:55:14.465 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-18T12:55:14.465 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-18T12:55:14.466 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-18T12:55:14.466 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-18T12:55:14.466 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-18T12:55:14.466 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-18T12:55:14.466 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-18T12:55:14.466 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-18T12:55:14.466 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-18T12:55:14.466 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-18T12:55:14.466 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-18T12:55:14.466 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-18T12:55:14.466 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-18T12:55:14.466 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:14.466 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:55:14.467 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:14.467 INFO:teuthology.orchestra.run.smithi179.stdout:Install 36 Packages 2024-06-18T12:55:14.467 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:14.467 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 30 M 2024-06-18T12:55:14.467 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 107 M 2024-06-18T12:55:14.467 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:55:15.455 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:19.0.0-3124.g3781e3a5.el9.x86_64 33/36 2024-06-18T12:55:15.482 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:19.0.0-3124.g3781e3a5.el9.x86_64 33/36 2024-06-18T12:55:15.482 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:55:15.482 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-18T12:55:15.482 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-18T12:55:15.482 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-18T12:55:15.482 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:15.534 INFO:teuthology.orchestra.run.smithi179.stdout:(1/36): ceph-19.0.0-3124.g3781e3a5.el9.x86_64.r 26 kB/s | 6.6 kB 00:00 2024-06-18T12:55:15.659 INFO:teuthology.orchestra.run.smithi179.stdout:(2/36): ceph-mds-19.0.0-3124.g3781e3a5.el9.x86_ 6.3 MB/s | 2.4 MB 00:00 2024-06-18T12:55:15.922 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:19.0.0-3124.g3781e3a5.el9.x86_64 34/36 2024-06-18T12:55:15.946 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:19.0.0-3124.g3781e3a5.el9.x86_64 34/36 2024-06-18T12:55:15.946 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:55:15.946 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-18T12:55:15.946 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-18T12:55:15.946 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-18T12:55:15.947 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:16.173 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:19.0.0-3124.g3781e3a5.el9.x86_64 35/36 2024-06-18T12:55:16.196 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:19.0.0-3124.g3781e3a5.el9.x86_64 35/36 2024-06-18T12:55:16.197 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:55:16.197 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-18T12:55:16.197 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-18T12:55:16.197 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-18T12:55:16.197 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:16.261 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:19.0.0-3124.g3781e3a5.el9.x86_64 36/36 2024-06-18T12:55:16.276 INFO:teuthology.orchestra.run.smithi179.stdout:(3/36): ceph-mgr-19.0.0-3124.g3781e3a5.el9.x86_ 1.5 MB/s | 1.5 MB 00:00 2024-06-18T12:55:16.477 INFO:teuthology.orchestra.run.smithi179.stdout:(4/36): ceph-osd-19.0.0-3124.g3781e3a5.el9.x86_ 21 MB/s | 17 MB 00:00 2024-06-18T12:55:16.510 INFO:teuthology.orchestra.run.smithi179.stdout:(5/36): ceph-mgr-modules-core-19.0.0-3124.g3781 1.0 MB/s | 249 kB 00:00 2024-06-18T12:55:16.685 INFO:teuthology.orchestra.run.smithi179.stdout:(6/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-06-18T12:55:16.744 INFO:teuthology.orchestra.run.smithi179.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-06-18T12:55:16.802 INFO:teuthology.orchestra.run.smithi179.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.3 MB/s | 135 kB 00:00 2024-06-18T12:55:16.861 INFO:teuthology.orchestra.run.smithi179.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 2.1 MB/s | 125 kB 00:00 2024-06-18T12:55:16.920 INFO:teuthology.orchestra.run.smithi179.stdout:(10/36): python3-cryptography-36.0.1-4.el9.x86_ 3.0 MB/s | 1.2 MB 00:00 2024-06-18T12:55:16.970 INFO:teuthology.orchestra.run.smithi179.stdout:(11/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.9 MB/s | 215 kB 00:00 2024-06-18T12:55:17.437 INFO:teuthology.orchestra.run.smithi179.stdout:(12/36): ceph-mon-19.0.0-3124.g3781e3a5.el9.x86 2.5 MB/s | 4.7 MB 00:01 2024-06-18T12:55:17.462 INFO:teuthology.orchestra.run.smithi179.stdout:(13/36): lua-5.4.4-4.el9.x86_64.rpm 347 kB/s | 188 kB 00:00 2024-06-18T12:55:17.489 INFO:teuthology.orchestra.run.smithi179.stdout:(14/36): python3-mako-1.1.4-6.el9.noarch.rpm 333 kB/s | 172 kB 00:00 2024-06-18T12:55:17.571 INFO:teuthology.orchestra.run.smithi179.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-06-18T12:55:17.596 INFO:teuthology.orchestra.run.smithi179.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-06-18T12:55:17.663 INFO:teuthology.orchestra.run.smithi179.stdout:(17/36): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-06-18T12:55:17.688 INFO:teuthology.orchestra.run.smithi179.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-06-18T12:55:17.714 INFO:teuthology.orchestra.run.smithi179.stdout:(19/36): python3-markupsafe-1.1.1-12.el9.x86_64 126 kB/s | 35 kB 00:00 2024-06-18T12:55:17.764 INFO:teuthology.orchestra.run.smithi179.stdout:(20/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 432 kB/s | 43 kB 00:00 2024-06-18T12:55:17.814 INFO:teuthology.orchestra.run.smithi179.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 492 kB/s | 11 kB 00:00 2024-06-18T12:55:17.839 INFO:teuthology.orchestra.run.smithi179.stdout:(22/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-06-18T12:55:17.865 INFO:teuthology.orchestra.run.smithi179.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 367 kB/s | 18 kB 00:00 2024-06-18T12:55:17.890 INFO:teuthology.orchestra.run.smithi179.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-06-18T12:55:17.915 INFO:teuthology.orchestra.run.smithi179.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-06-18T12:55:17.941 INFO:teuthology.orchestra.run.smithi179.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-06-18T12:55:17.966 INFO:teuthology.orchestra.run.smithi179.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-06-18T12:55:17.991 INFO:teuthology.orchestra.run.smithi179.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-06-18T12:55:18.017 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-2:19.0.0-3124.g3781e3a5.el9.x86_64 36/36 2024-06-18T12:55:18.017 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/36 2024-06-18T12:55:18.018 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:19.0.0-3124.g3781e3a5.el9.x86_64 2/36 2024-06-18T12:55:18.018 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:19.0.0-3124.g3781e3a5.el9.x86_64 3/36 2024-06-18T12:55:18.018 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:19.0.0-3124.g3781e3a5.el9.x86_64 4/36 2024-06-18T12:55:18.018 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:19.0.0-3124.g3781e3a5.el9.x86_64 5/36 2024-06-18T12:55:18.018 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3124.g3781e3a5.el9. 6/36 2024-06-18T12:55:18.018 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-06-18T12:55:18.018 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-06-18T12:55:18.018 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-06-18T12:55:18.018 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-06-18T12:55:18.018 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-06-18T12:55:18.018 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-06-18T12:55:18.018 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-06-18T12:55:18.019 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-06-18T12:55:18.019 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-06-18T12:55:18.019 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-06-18T12:55:18.019 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-06-18T12:55:18.019 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-06-18T12:55:18.019 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 19/36 2024-06-18T12:55:18.019 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-06-18T12:55:18.019 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-06-18T12:55:18.019 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-06-18T12:55:18.019 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-06-18T12:55:18.019 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-06-18T12:55:18.019 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-06-18T12:55:18.019 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-06-18T12:55:18.019 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-06-18T12:55:18.020 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-06-18T12:55:18.020 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-06-18T12:55:18.020 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-06-18T12:55:18.020 INFO:teuthology.orchestra.run.smithi179.stdout:(29/36): python3-portend-3.1.0-2.el9.noarch.rpm 661 kB/s | 16 kB 00:00 2024-06-18T12:55:18.021 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-06-18T12:55:18.021 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-06-18T12:55:18.021 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-06-18T12:55:18.021 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-06-18T12:55:18.021 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-06-18T12:55:18.042 INFO:teuthology.orchestra.run.smithi179.stdout:(30/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-06-18T12:55:18.067 INFO:teuthology.orchestra.run.smithi179.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-06-18T12:55:18.092 INFO:teuthology.orchestra.run.smithi179.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-06-18T12:55:18.117 INFO:teuthology.orchestra.run.smithi179.stdout:(33/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-06-18T12:55:18.143 INFO:teuthology.orchestra.run.smithi179.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-06-18T12:55:18.177 INFO:teuthology.orchestra.run.smithi179.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 339 kB/s | 20 kB 00:00 2024-06-18T12:55:18.202 INFO:teuthology.orchestra.run.smithi179.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-06-18T12:55:18.207 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:55:18.207 INFO:teuthology.orchestra.run.smithi179.stdout:Total 8.1 MB/s | 30 MB 00:03 2024-06-18T12:55:18.348 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:55:18.387 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:55:18.387 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:55:18.829 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:55:18.830 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:55:19.075 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-06-18T12:55:19.075 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:19.075 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:55:19.076 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:19.076 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:19.076 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:19.076 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:55:19.076 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:19.076 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:19.076 INFO:teuthology.orchestra.run.smithi167.stdout: lua-5.4.4-4.el9.x86_64 2024-06-18T12:55:19.076 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-18T12:55:19.076 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-18T12:55:19.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-18T12:55:19.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-18T12:55:19.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-18T12:55:19.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-18T12:55:19.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-18T12:55:19.077 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-18T12:55:19.077 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-18T12:55:19.077 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-18T12:55:19.077 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-18T12:55:19.077 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-18T12:55:19.077 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-18T12:55:19.077 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-18T12:55:19.077 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-18T12:55:19.077 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-18T12:55:19.077 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-18T12:55:19.077 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply-3.11-14.el9.noarch 2024-06-18T12:55:19.077 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-18T12:55:19.077 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-18T12:55:19.078 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-18T12:55:19.078 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-18T12:55:19.078 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-18T12:55:19.078 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-18T12:55:19.078 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-18T12:55:19.078 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-18T12:55:19.078 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-18T12:55:19.078 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-18T12:55:19.078 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-18T12:55:19.078 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:19.078 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:55:19.445 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-base 2024-06-18T12:55:20.031 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:54 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:55:20.114 INFO:teuthology.orchestra.run.smithi167.stdout:Package ceph-base-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:55:20.118 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:55:20.155 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:55:20.156 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2024-06-18T12:55:20.156 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:55:20.196 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-06-18T12:55:20.210 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install cephadm 2024-06-18T12:55:20.229 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-06-18T12:55:20.257 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-06-18T12:55:20.330 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-06-18T12:55:20.373 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/36 2024-06-18T12:55:20.405 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-06-18T12:55:20.497 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-06-18T12:55:20.593 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-06-18T12:55:20.633 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-06-18T12:55:20.668 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-06-18T12:55:20.698 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-06-18T12:55:20.732 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-06-18T12:55:20.766 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-06-18T12:55:20.780 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:54 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:55:20.806 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-06-18T12:55:20.841 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-06-18T12:55:20.876 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-06-18T12:55:20.904 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:55:20.905 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:55:20.905 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-06-18T12:55:20.905 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:55:20.905 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:55:20.905 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 762 k 2024-06-18T12:55:20.905 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:20.905 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:55:20.905 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:55:20.905 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2024-06-18T12:55:20.905 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:20.905 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 762 k 2024-06-18T12:55:20.905 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 769 k 2024-06-18T12:55:20.906 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:55:20.906 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-06-18T12:55:20.991 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-06-18T12:55:21.134 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-06-18T12:55:21.214 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-06-18T12:55:21.267 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-06-18T12:55:21.306 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-06-18T12:55:21.363 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-06-18T12:55:21.372 INFO:teuthology.orchestra.run.smithi167.stdout:cephadm-19.0.0-3124.g3781e3a5.el9.noarch.rpm 1.6 MB/s | 762 kB 00:00 2024-06-18T12:55:21.373 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:55:21.373 INFO:teuthology.orchestra.run.smithi167.stdout:Total 1.6 MB/s | 762 kB 00:00 2024-06-18T12:55:21.374 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:55:21.379 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:55:21.379 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:55:21.385 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:55:21.385 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:55:21.427 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-06-18T12:55:21.467 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:55:21.612 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-06-18T12:55:21.659 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-06-18T12:55:21.738 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-06-18T12:55:21.827 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-06-18T12:55:21.873 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-06-18T12:55:21.967 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 1/1 2024-06-18T12:55:21.991 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-06-18T12:55:22.068 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 1/1 2024-06-18T12:55:22.190 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3124.g3781e3a5.el9. 31/36 2024-06-18T12:55:22.227 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-2:19.0.0-3124.g3781e3a5.el9.x86_64 32/36 2024-06-18T12:55:22.253 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3124.g3781e3a5.el9.x86_64 32/36 2024-06-18T12:55:22.253 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:55:22.253 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-18T12:55:22.253 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-18T12:55:22.253 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-18T12:55:22.253 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:22.420 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 1/1 2024-06-18T12:55:22.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 1/1 2024-06-18T12:55:22.656 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:22.656 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:55:22.656 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:55:22.656 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:22.656 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:55:22.918 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-immutable-object-cache 2024-06-18T12:55:23.526 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:57 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:55:23.657 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:55:23.658 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:55:23.659 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repo Size 2024-06-18T12:55:23.659 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:55:23.659 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:55:23.659 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 145 k 2024-06-18T12:55:23.659 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:23.659 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:55:23.659 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:55:23.659 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2024-06-18T12:55:23.659 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:23.659 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 145 k 2024-06-18T12:55:23.660 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 435 k 2024-06-18T12:55:23.660 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:55:23.687 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-osd-2:19.0.0-3124.g3781e3a5.el9.x86_64 33/36 2024-06-18T12:55:23.714 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-osd-2:19.0.0-3124.g3781e3a5.el9.x86_64 33/36 2024-06-18T12:55:23.714 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:55:23.714 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-18T12:55:23.714 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-18T12:55:23.714 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-18T12:55:23.714 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:24.022 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-immutable-object-cache-19.0.0-3124.g3781e3 399 kB/s | 145 kB 00:00 2024-06-18T12:55:24.023 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:55:24.023 INFO:teuthology.orchestra.run.smithi167.stdout:Total 397 kB/s | 145 kB 00:00 2024-06-18T12:55:24.023 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:55:24.031 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:55:24.031 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:55:24.088 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:55:24.089 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:55:24.162 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mon-2:19.0.0-3124.g3781e3a5.el9.x86_64 34/36 2024-06-18T12:55:24.186 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mon-2:19.0.0-3124.g3781e3a5.el9.x86_64 34/36 2024-06-18T12:55:24.186 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:55:24.186 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-18T12:55:24.186 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-18T12:55:24.186 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-18T12:55:24.186 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:24.226 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:55:24.290 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3124.g3781e3a5. 1/1 2024-06-18T12:55:24.316 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3124.g3781e3a5. 1/1 2024-06-18T12:55:24.316 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:55:24.316 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-18T12:55:24.316 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:24.405 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mds-2:19.0.0-3124.g3781e3a5.el9.x86_64 35/36 2024-06-18T12:55:24.428 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mds-2:19.0.0-3124.g3781e3a5.el9.x86_64 35/36 2024-06-18T12:55:24.429 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:55:24.429 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-18T12:55:24.429 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-18T12:55:24.429 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-18T12:55:24.429 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:24.495 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-2:19.0.0-3124.g3781e3a5.el9.x86_64 36/36 2024-06-18T12:55:25.245 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3124.g3781e3a5. 1/1 2024-06-18T12:55:25.245 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:25.245 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:55:25.246 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:25.246 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:25.246 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:55:25.458 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr 2024-06-18T12:55:26.047 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:00 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:55:26.135 INFO:teuthology.orchestra.run.smithi167.stdout:Package ceph-mgr-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:55:26.176 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:55:26.177 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2024-06-18T12:55:26.178 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:55:26.237 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr-dashboard 2024-06-18T12:55:26.283 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-2:19.0.0-3124.g3781e3a5.el9.x86_64 36/36 2024-06-18T12:55:26.283 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/36 2024-06-18T12:55:26.283 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mds-2:19.0.0-3124.g3781e3a5.el9.x86_64 2/36 2024-06-18T12:55:26.283 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-2:19.0.0-3124.g3781e3a5.el9.x86_64 3/36 2024-06-18T12:55:26.283 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mon-2:19.0.0-3124.g3781e3a5.el9.x86_64 4/36 2024-06-18T12:55:26.283 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-osd-2:19.0.0-3124.g3781e3a5.el9.x86_64 5/36 2024-06-18T12:55:26.283 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3124.g3781e3a5.el9. 6/36 2024-06-18T12:55:26.283 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-06-18T12:55:26.283 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-06-18T12:55:26.283 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-06-18T12:55:26.284 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-06-18T12:55:26.284 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-06-18T12:55:26.284 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-06-18T12:55:26.284 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-06-18T12:55:26.284 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-06-18T12:55:26.284 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-06-18T12:55:26.284 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-06-18T12:55:26.284 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-06-18T12:55:26.284 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-06-18T12:55:26.284 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 19/36 2024-06-18T12:55:26.284 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-06-18T12:55:26.284 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-06-18T12:55:26.284 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-06-18T12:55:26.284 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-06-18T12:55:26.285 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-06-18T12:55:26.285 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-06-18T12:55:26.286 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-06-18T12:55:26.286 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-06-18T12:55:26.286 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-06-18T12:55:26.286 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-06-18T12:55:26.286 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-06-18T12:55:26.286 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-06-18T12:55:26.286 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-06-18T12:55:26.286 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-06-18T12:55:26.286 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-06-18T12:55:26.286 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-06-18T12:55:26.807 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:00 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:55:26.933 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:55:26.934 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:55:26.934 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-06-18T12:55:26.935 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:55:26.935 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:55:26.935 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 3.5 M 2024-06-18T12:55:26.935 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-06-18T12:55:26.935 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-18T12:55:26.935 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 32 k 2024-06-18T12:55:26.935 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 16 k 2024-06-18T12:55:26.935 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-18T12:55:26.935 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-18T12:55:26.935 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-18T12:55:26.935 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-18T12:55:26.935 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-18T12:55:26.936 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-18T12:55:26.936 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-18T12:55:26.936 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-18T12:55:26.936 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:26.936 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:55:26.936 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:55:26.936 INFO:teuthology.orchestra.run.smithi167.stdout:Install 12 Packages 2024-06-18T12:55:26.936 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:26.937 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 8.6 M 2024-06-18T12:55:26.937 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 103 M 2024-06-18T12:55:26.937 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:55:27.216 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-06-18T12:55:27.217 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:27.217 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:55:27.217 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:27.217 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:27.217 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:27.217 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:55:27.217 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:27.217 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:27.217 INFO:teuthology.orchestra.run.smithi179.stdout: lua-5.4.4-4.el9.x86_64 2024-06-18T12:55:27.217 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-18T12:55:27.217 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-18T12:55:27.218 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-18T12:55:27.218 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-18T12:55:27.218 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-18T12:55:27.218 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-18T12:55:27.218 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-18T12:55:27.218 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-18T12:55:27.218 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-18T12:55:27.218 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-18T12:55:27.218 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-18T12:55:27.218 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-18T12:55:27.218 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-18T12:55:27.218 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-18T12:55:27.218 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-18T12:55:27.218 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply-3.11-14.el9.noarch 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:27.219 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:55:27.545 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-base 2024-06-18T12:55:27.843 INFO:teuthology.orchestra.run.smithi167.stdout:(1/12): ceph-prometheus-alerts-19.0.0-3124.g378 86 kB/s | 16 kB 00:00 2024-06-18T12:55:27.951 INFO:teuthology.orchestra.run.smithi167.stdout:(2/12): ceph-grafana-dashboards-19.0.0-3124.g37 106 kB/s | 32 kB 00:00 2024-06-18T12:55:28.076 INFO:teuthology.orchestra.run.smithi167.stdout:(3/12): ceph-mgr-dashboard-19.0.0-3124.g3781e3a 8.3 MB/s | 3.5 MB 00:00 2024-06-18T12:55:28.127 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:54 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:55:28.211 INFO:teuthology.orchestra.run.smithi179.stdout:Package ceph-base-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:55:28.252 INFO:teuthology.orchestra.run.smithi167.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 897 kB/s | 269 kB 00:00 2024-06-18T12:55:28.252 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:55:28.253 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-06-18T12:55:28.253 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:55:28.285 INFO:teuthology.orchestra.run.smithi167.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.3 MB/s | 1.0 MB 00:00 2024-06-18T12:55:28.301 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install cephadm 2024-06-18T12:55:28.377 INFO:teuthology.orchestra.run.smithi167.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-06-18T12:55:28.427 INFO:teuthology.orchestra.run.smithi167.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.0 MB/s | 548 kB 00:00 2024-06-18T12:55:28.461 INFO:teuthology.orchestra.run.smithi167.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-06-18T12:55:28.486 INFO:teuthology.orchestra.run.smithi167.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-06-18T12:55:28.520 INFO:teuthology.orchestra.run.smithi167.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.7 MB/s | 188 kB 00:00 2024-06-18T12:55:28.570 INFO:teuthology.orchestra.run.smithi167.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 12 MB/s | 2.0 MB 00:00 2024-06-18T12:55:28.872 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:54 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:55:28.998 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:55:28.998 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:28.998 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-06-18T12:55:28.999 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:28.999 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:55:28.999 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 762 k 2024-06-18T12:55:28.999 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:28.999 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:55:28.999 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:28.999 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-06-18T12:55:28.999 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:28.999 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 762 k 2024-06-18T12:55:28.999 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 769 k 2024-06-18T12:55:28.999 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:55:29.162 INFO:teuthology.orchestra.run.smithi167.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 795 kB/s | 863 kB 00:01 2024-06-18T12:55:29.167 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:55:29.167 INFO:teuthology.orchestra.run.smithi167.stdout:Total 3.9 MB/s | 8.6 MB 00:02 2024-06-18T12:55:29.215 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:55:29.231 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:55:29.231 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:55:29.420 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:55:29.420 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:55:29.446 INFO:teuthology.orchestra.run.smithi179.stdout:cephadm-19.0.0-3124.g3781e3a5.el9.noarch.rpm 1.7 MB/s | 762 kB 00:00 2024-06-18T12:55:29.446 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:55:29.447 INFO:teuthology.orchestra.run.smithi179.stdout:Total 1.7 MB/s | 762 kB 00:00 2024-06-18T12:55:29.447 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:55:29.452 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:55:29.452 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:55:29.458 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:55:29.458 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:55:29.541 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:55:29.944 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:55:29.947 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 1/1 2024-06-18T12:55:30.030 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-06-18T12:55:30.032 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 1/1 2024-06-18T12:55:30.073 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-06-18T12:55:30.120 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-06-18T12:55:30.159 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-06-18T12:55:30.264 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-06-18T12:55:30.382 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 1/1 2024-06-18T12:55:30.393 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-06-18T12:55:30.487 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-06-18T12:55:30.530 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-06-18T12:55:30.558 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-06-18T12:55:30.590 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3124.g3781e3a5.el9 10/12 2024-06-18T12:55:30.620 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 1/1 2024-06-18T12:55:30.620 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:30.621 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:55:30.621 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:55:30.621 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:30.621 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:55:30.835 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-immutable-object-cache 2024-06-18T12:55:31.423 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:57 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:55:31.554 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:55:31.555 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:31.555 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repo Size 2024-06-18T12:55:31.555 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:31.555 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:55:31.555 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 145 k 2024-06-18T12:55:31.555 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:31.555 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:55:31.556 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:31.556 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-06-18T12:55:31.556 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:31.556 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 145 k 2024-06-18T12:55:31.556 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 435 k 2024-06-18T12:55:31.556 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:55:31.629 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3124.g3781e3a5.el 11/12 2024-06-18T12:55:31.683 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3124.g3781e3a5.el9.noa 12/12 2024-06-18T12:55:31.961 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-immutable-object-cache-19.0.0-3124.g3781e3 357 kB/s | 145 kB 00:00 2024-06-18T12:55:31.961 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:55:31.961 INFO:teuthology.orchestra.run.smithi179.stdout:Total 356 kB/s | 145 kB 00:00 2024-06-18T12:55:31.962 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:55:31.969 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:55:31.969 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:55:32.027 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:55:32.027 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:55:32.168 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:55:32.245 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3124.g3781e3a5. 1/1 2024-06-18T12:55:32.271 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3124.g3781e3a5. 1/1 2024-06-18T12:55:32.271 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:55:32.271 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-18T12:55:32.271 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:33.033 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3124.g3781e3a5.el9.noa 12/12 2024-06-18T12:55:33.033 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3124.g3781e3a5.el 1/12 2024-06-18T12:55:33.033 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3124.g3781e3a5.el9.noa 2/12 2024-06-18T12:55:33.033 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3124.g3781e3a5.el9 3/12 2024-06-18T12:55:33.033 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-06-18T12:55:33.033 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-06-18T12:55:33.033 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-06-18T12:55:33.033 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-06-18T12:55:33.034 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-06-18T12:55:33.034 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-06-18T12:55:33.034 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-06-18T12:55:33.034 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-06-18T12:55:33.187 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3124.g3781e3a5. 1/1 2024-06-18T12:55:33.187 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:33.187 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:55:33.187 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:55:33.187 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:33.187 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:55:33.344 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-06-18T12:55:33.344 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:33.344 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:55:33.344 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-18T12:55:33.344 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:55:33.345 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:55:33.345 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:55:33.345 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-18T12:55:33.345 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-18T12:55:33.345 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-18T12:55:33.345 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-18T12:55:33.345 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-18T12:55:33.345 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-18T12:55:33.345 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-18T12:55:33.345 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-18T12:55:33.345 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:33.345 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:55:33.397 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr 2024-06-18T12:55:33.619 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr-diskprediction-local 2024-06-18T12:55:33.987 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:59 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:55:34.075 INFO:teuthology.orchestra.run.smithi179.stdout:Package ceph-mgr-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:55:34.116 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:55:34.117 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-06-18T12:55:34.117 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:55:34.180 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-dashboard 2024-06-18T12:55:34.198 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:08 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:55:34.327 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:55:34.328 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-06-18T12:55:34.328 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-06-18T12:55:34.329 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-06-18T12:55:34.329 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:55:34.329 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 7.4 M 2024-06-18T12:55:34.329 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-06-18T12:55:34.329 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-18T12:55:34.329 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-18T12:55:34.329 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-18T12:55:34.329 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-18T12:55:34.329 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-18T12:55:34.329 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-18T12:55:34.329 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-18T12:55:34.329 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-18T12:55:34.329 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-18T12:55:34.329 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-18T12:55:34.330 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-18T12:55:34.330 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:34.330 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:55:34.330 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-06-18T12:55:34.330 INFO:teuthology.orchestra.run.smithi167.stdout:Install 12 Packages 2024-06-18T12:55:34.330 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:55:34.330 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 38 M 2024-06-18T12:55:34.330 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 209 M 2024-06-18T12:55:34.331 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:55:34.747 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:00 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:55:34.862 INFO:teuthology.orchestra.run.smithi167.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 811 kB/s | 195 kB 00:00 2024-06-18T12:55:34.872 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:55:34.873 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:34.873 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-06-18T12:55:34.873 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:34.873 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:55:34.873 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 3.5 M 2024-06-18T12:55:34.873 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-06-18T12:55:34.873 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-18T12:55:34.874 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 32 k 2024-06-18T12:55:34.874 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 16 k 2024-06-18T12:55:34.874 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-18T12:55:34.874 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-18T12:55:34.874 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-18T12:55:34.874 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-18T12:55:34.874 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-18T12:55:34.874 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-18T12:55:34.874 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-18T12:55:34.874 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-18T12:55:34.874 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:34.874 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:55:34.874 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:55:34.875 INFO:teuthology.orchestra.run.smithi179.stdout:Install 12 Packages 2024-06-18T12:55:34.875 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:34.875 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 8.6 M 2024-06-18T12:55:34.875 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 103 M 2024-06-18T12:55:34.875 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:55:34.954 INFO:teuthology.orchestra.run.smithi167.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-06-18T12:55:35.138 INFO:teuthology.orchestra.run.smithi167.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 119 kB/s | 33 kB 00:00 2024-06-18T12:55:35.246 INFO:teuthology.orchestra.run.smithi167.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 168 kB/s | 18 kB 00:00 2024-06-18T12:55:35.390 INFO:teuthology.orchestra.run.smithi179.stdout:(1/12): ceph-grafana-dashboards-19.0.0-3124.g37 120 kB/s | 32 kB 00:00 2024-06-18T12:55:35.415 INFO:teuthology.orchestra.run.smithi179.stdout:(2/12): ceph-prometheus-alerts-19.0.0-3124.g378 57 kB/s | 16 kB 00:00 2024-06-18T12:55:35.430 INFO:teuthology.orchestra.run.smithi167.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 192 kB/s | 35 kB 00:00 2024-06-18T12:55:35.598 INFO:teuthology.orchestra.run.smithi179.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.4 MB/s | 269 kB 00:00 2024-06-18T12:55:35.657 INFO:teuthology.orchestra.run.smithi179.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.8 MB/s | 1.0 MB 00:00 2024-06-18T12:55:35.749 INFO:teuthology.orchestra.run.smithi179.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 5.9 MB/s | 548 kB 00:00 2024-06-18T12:55:35.774 INFO:teuthology.orchestra.run.smithi179.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 774 kB/s | 19 kB 00:00 2024-06-18T12:55:35.866 INFO:teuthology.orchestra.run.smithi179.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 22 MB/s | 2.0 MB 00:00 2024-06-18T12:55:35.900 INFO:teuthology.orchestra.run.smithi179.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.3 MB/s | 144 kB 00:00 2024-06-18T12:55:35.925 INFO:teuthology.orchestra.run.smithi179.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-06-18T12:55:35.950 INFO:teuthology.orchestra.run.smithi179.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.4 MB/s | 188 kB 00:00 2024-06-18T12:55:35.997 INFO:teuthology.orchestra.run.smithi167.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.9 MB/s | 3.0 MB 00:01 2024-06-18T12:55:36.139 INFO:teuthology.orchestra.run.smithi167.stdout:(7/12): ceph-mgr-diskprediction-local-19.0.0-31 4.9 MB/s | 7.4 MB 00:01 2024-06-18T12:55:36.189 INFO:teuthology.orchestra.run.smithi167.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.2 MB/s | 244 kB 00:00 2024-06-18T12:55:36.267 INFO:teuthology.orchestra.run.smithi179.stdout:(11/12): ceph-mgr-dashboard-19.0.0-3124.g3781e3 3.1 MB/s | 3.5 MB 00:01 2024-06-18T12:55:36.356 INFO:teuthology.orchestra.run.smithi167.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 5.1 MB/s | 4.7 MB 00:00 2024-06-18T12:55:36.384 INFO:teuthology.orchestra.run.smithi179.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.1 MB/s | 863 kB 00:00 2024-06-18T12:55:36.388 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:55:36.388 INFO:teuthology.orchestra.run.smithi179.stdout:Total 5.7 MB/s | 8.6 MB 00:01 2024-06-18T12:55:36.436 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:55:36.453 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:55:36.453 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:55:36.473 INFO:teuthology.orchestra.run.smithi167.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 883 kB/s | 250 kB 00:00 2024-06-18T12:55:36.624 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:55:36.625 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:55:37.143 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:55:37.219 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-06-18T12:55:37.262 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-06-18T12:55:37.308 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-06-18T12:55:37.347 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-06-18T12:55:37.453 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-06-18T12:55:37.573 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-06-18T12:55:37.666 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-06-18T12:55:37.710 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-06-18T12:55:37.739 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-06-18T12:55:37.770 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3124.g3781e3a5.el9 10/12 2024-06-18T12:55:37.981 INFO:teuthology.orchestra.run.smithi167.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.8 MB/s | 5.1 MB 00:01 2024-06-18T12:55:38.805 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3124.g3781e3a5.el 11/12 2024-06-18T12:55:38.838 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3124.g3781e3a5.el9.noa 12/12 2024-06-18T12:55:40.191 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3124.g3781e3a5.el9.noa 12/12 2024-06-18T12:55:40.192 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3124.g3781e3a5.el 1/12 2024-06-18T12:55:40.192 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3124.g3781e3a5.el9.noa 2/12 2024-06-18T12:55:40.192 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3124.g3781e3a5.el9 3/12 2024-06-18T12:55:40.192 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-06-18T12:55:40.192 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-06-18T12:55:40.192 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-06-18T12:55:40.192 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-06-18T12:55:40.192 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-06-18T12:55:40.192 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-06-18T12:55:40.192 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-06-18T12:55:40.192 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-06-18T12:55:40.502 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-06-18T12:55:40.502 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:40.502 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:55:40.502 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-18T12:55:40.503 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:55:40.503 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:55:40.503 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:55:40.503 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-18T12:55:40.503 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-18T12:55:40.503 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-18T12:55:40.503 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-18T12:55:40.503 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-18T12:55:40.503 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-18T12:55:40.503 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-18T12:55:40.503 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-18T12:55:40.503 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:40.503 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:55:40.758 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-diskprediction-local 2024-06-18T12:55:41.329 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:07 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:55:41.459 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:55:41.460 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-06-18T12:55:41.460 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-06-18T12:55:41.460 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-06-18T12:55:41.460 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:55:41.460 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 7.4 M 2024-06-18T12:55:41.460 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-06-18T12:55:41.460 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-18T12:55:41.460 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-18T12:55:41.460 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-18T12:55:41.460 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-18T12:55:41.460 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-18T12:55:41.460 INFO:teuthology.orchestra.run.smithi179.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-18T12:55:41.461 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-18T12:55:41.461 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-18T12:55:41.461 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-18T12:55:41.461 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-18T12:55:41.461 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-18T12:55:41.461 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:41.461 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:55:41.461 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-06-18T12:55:41.461 INFO:teuthology.orchestra.run.smithi179.stdout:Install 12 Packages 2024-06-18T12:55:41.461 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:41.462 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 38 M 2024-06-18T12:55:41.462 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 209 M 2024-06-18T12:55:41.462 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:55:41.817 INFO:teuthology.orchestra.run.smithi179.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-06-18T12:55:41.876 INFO:teuthology.orchestra.run.smithi179.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.0 MB/s | 803 kB 00:00 2024-06-18T12:55:42.085 INFO:teuthology.orchestra.run.smithi179.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 122 kB/s | 33 kB 00:00 2024-06-18T12:55:42.202 INFO:teuthology.orchestra.run.smithi179.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 156 kB/s | 18 kB 00:00 2024-06-18T12:55:42.393 INFO:teuthology.orchestra.run.smithi179.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 184 kB/s | 35 kB 00:00 2024-06-18T12:55:42.835 INFO:teuthology.orchestra.run.smithi179.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.1 MB/s | 3.0 MB 00:00 2024-06-18T12:55:42.952 INFO:teuthology.orchestra.run.smithi179.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 2.0 MB/s | 244 kB 00:00 2024-06-18T12:55:43.136 INFO:teuthology.orchestra.run.smithi179.stdout:(8/12): ceph-mgr-diskprediction-local-19.0.0-31 5.1 MB/s | 7.4 MB 00:01 2024-06-18T12:55:43.720 INFO:teuthology.orchestra.run.smithi179.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 430 kB/s | 250 kB 00:00 2024-06-18T12:55:44.995 INFO:teuthology.orchestra.run.smithi179.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 1.8 MB/s | 4.7 MB 00:02 2024-06-18T12:55:48.195 INFO:teuthology.orchestra.run.smithi179.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 996 kB/s | 5.1 MB 00:05 2024-06-18T12:55:51.636 INFO:teuthology.orchestra.run.smithi179.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.1 MB/s | 16 MB 00:07 2024-06-18T12:55:51.637 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:55:51.638 INFO:teuthology.orchestra.run.smithi179.stdout:Total 3.7 MB/s | 38 MB 00:10 2024-06-18T12:55:51.995 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:55:52.023 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:55:52.023 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:55:52.449 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:55:52.450 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:55:53.202 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:55:53.270 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-06-18T12:55:53.309 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-06-18T12:55:53.417 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-06-18T12:55:53.460 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-06-18T12:55:53.637 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-06-18T12:55:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-06-18T12:55:53.744 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-06-18T12:55:54.398 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-06-18T12:55:54.514 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-06-18T12:55:55.747 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-06-18T12:55:56.613 INFO:teuthology.orchestra.run.smithi167.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 822 kB/s | 16 MB 00:20 2024-06-18T12:55:56.614 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:55:56.614 INFO:teuthology.orchestra.run.smithi167.stdout:Total 1.7 MB/s | 38 MB 00:22 2024-06-18T12:55:56.789 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-06-18T12:55:56.842 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3124.g3781e 12/12 2024-06-18T12:55:56.968 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:55:56.998 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:55:56.998 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:55:57.426 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:55:57.426 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:55:58.184 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:55:58.266 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-06-18T12:55:58.303 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-06-18T12:55:58.405 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-06-18T12:55:58.445 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-06-18T12:55:58.622 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-06-18T12:55:58.652 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-06-18T12:55:58.729 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-06-18T12:55:59.407 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-06-18T12:55:59.457 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3124.g3781e 12/12 2024-06-18T12:55:59.457 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3124.g3781e 1/12 2024-06-18T12:55:59.457 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-06-18T12:55:59.457 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-06-18T12:55:59.457 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-06-18T12:55:59.457 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-06-18T12:55:59.457 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-06-18T12:55:59.457 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-06-18T12:55:59.457 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-06-18T12:55:59.457 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-06-18T12:55:59.457 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-06-18T12:55:59.458 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-06-18T12:55:59.533 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-06-18T12:55:59.859 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-06-18T12:55:59.859 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:59.859 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:55:59.859 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:55:59.859 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-18T12:55:59.859 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-18T12:55:59.859 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-18T12:55:59.859 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-18T12:55:59.859 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-18T12:55:59.859 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-18T12:55:59.859 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-18T12:55:59.859 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-18T12:55:59.860 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-18T12:55:59.860 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-18T12:55:59.860 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-18T12:55:59.860 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:55:59.860 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:00.272 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-rook 2024-06-18T12:56:00.789 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-06-18T12:56:00.858 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:26 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:00.984 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:00.985 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:00.986 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-06-18T12:56:00.986 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:00.986 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:56:00.986 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 49 k 2024-06-18T12:56:00.986 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-06-18T12:56:00.986 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-18T12:56:00.986 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-18T12:56:00.986 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth noarch 1:2.30.0-1.el9 epel 213 k 2024-06-18T12:56:00.986 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-18T12:56:00.986 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-18T12:56:00.986 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-18T12:56:00.987 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-18T12:56:00.987 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-18T12:56:00.987 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-18T12:56:00.987 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-18T12:56:00.987 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-18T12:56:00.987 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-18T12:56:00.987 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:00.987 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:56:00.987 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:00.987 INFO:teuthology.orchestra.run.smithi179.stdout:Install 13 Packages 2024-06-18T12:56:00.987 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:00.988 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 2.2 M 2024-06-18T12:56:00.988 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 26 M 2024-06-18T12:56:00.988 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:56:01.528 INFO:teuthology.orchestra.run.smithi179.stdout:(1/13): ceph-mgr-rook-19.0.0-3124.g3781e3a5.el9 157 kB/s | 49 kB 00:00 2024-06-18T12:56:01.595 INFO:teuthology.orchestra.run.smithi179.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 67 kB/s | 26 kB 00:00 2024-06-18T12:56:01.620 INFO:teuthology.orchestra.run.smithi179.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 47 kB/s | 19 kB 00:00 2024-06-18T12:56:01.817 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-06-18T12:56:01.820 INFO:teuthology.orchestra.run.smithi179.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 760 kB/s | 222 kB 00:00 2024-06-18T12:56:01.850 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3124.g3781e 12/12 2024-06-18T12:56:01.945 INFO:teuthology.orchestra.run.smithi179.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 453 kB/s | 159 kB 00:00 2024-06-18T12:56:01.970 INFO:teuthology.orchestra.run.smithi179.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 795 kB/s | 279 kB 00:00 2024-06-18T12:56:01.996 INFO:teuthology.orchestra.run.smithi179.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 306 kB/s | 54 kB 00:00 2024-06-18T12:56:02.096 INFO:teuthology.orchestra.run.smithi179.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 215 kB/s | 32 kB 00:00 2024-06-18T12:56:02.121 INFO:teuthology.orchestra.run.smithi179.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 120 kB/s | 15 kB 00:00 2024-06-18T12:56:02.155 INFO:teuthology.orchestra.run.smithi179.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-06-18T12:56:02.180 INFO:teuthology.orchestra.run.smithi179.stdout:(11/13): python3-google-auth-2.30.0-1.el9.noarc 1.1 MB/s | 213 kB 00:00 2024-06-18T12:56:02.205 INFO:teuthology.orchestra.run.smithi179.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-06-18T12:56:02.239 INFO:teuthology.orchestra.run.smithi179.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.3 MB/s | 1.0 MB 00:00 2024-06-18T12:56:02.243 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:02.243 INFO:teuthology.orchestra.run.smithi179.stdout:Total 1.8 MB/s | 2.2 MB 00:01 2024-06-18T12:56:02.307 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:56:02.322 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:56:02.322 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:56:02.409 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:56:02.410 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:56:02.925 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:56:03.114 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-06-18T12:56:03.166 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-06-18T12:56:03.221 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-06-18T12:56:03.258 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-06-18T12:56:03.300 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-06-18T12:56:03.398 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-06-18T12:56:03.514 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-google-auth-1:2.30.0-1.el9.noarch 7/13 2024-06-18T12:56:03.568 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-06-18T12:56:04.304 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-06-18T12:56:04.349 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-06-18T12:56:04.382 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-06-18T12:56:04.389 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3124.g3781e 12/12 2024-06-18T12:56:04.389 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3124.g3781e 1/12 2024-06-18T12:56:04.389 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-06-18T12:56:04.389 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-06-18T12:56:04.389 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-06-18T12:56:04.389 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-06-18T12:56:04.389 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-06-18T12:56:04.389 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-06-18T12:56:04.390 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-06-18T12:56:04.390 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-06-18T12:56:04.390 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-06-18T12:56:04.390 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-06-18T12:56:04.439 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-06-18T12:56:04.469 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-rook-2:19.0.0-3124.g3781e3a5.el9.noarch 13/13 2024-06-18T12:56:04.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-06-18T12:56:04.784 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:04.785 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:56:04.785 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:56:04.785 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-18T12:56:04.785 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-18T12:56:04.785 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-18T12:56:04.785 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-18T12:56:04.785 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-18T12:56:04.785 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-18T12:56:04.785 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-18T12:56:04.785 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-18T12:56:04.785 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-18T12:56:04.785 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-18T12:56:04.786 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-18T12:56:04.786 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:04.786 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:05.240 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr-rook 2024-06-18T12:56:05.274 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3124.g3781e3a5.el9.noarch 13/13 2024-06-18T12:56:05.274 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3124.g3781e3a5.el9.noarch 1/13 2024-06-18T12:56:05.274 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-06-18T12:56:05.274 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-06-18T12:56:05.274 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-06-18T12:56:05.274 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-06-18T12:56:05.275 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-06-18T12:56:05.275 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-06-18T12:56:05.275 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-06-18T12:56:05.275 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-06-18T12:56:05.275 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-google-auth-1:2.30.0-1.el9.noarch 10/13 2024-06-18T12:56:05.275 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-06-18T12:56:05.275 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-06-18T12:56:05.599 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-06-18T12:56:05.600 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:05.600 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:56:05.600 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:56:05.600 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-18T12:56:05.600 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-18T12:56:05.600 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth-1:2.30.0-1.el9.noarch 2024-06-18T12:56:05.600 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-18T12:56:05.600 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-18T12:56:05.600 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-18T12:56:05.600 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-18T12:56:05.600 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-18T12:56:05.600 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-18T12:56:05.600 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-18T12:56:05.601 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-18T12:56:05.601 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-18T12:56:05.601 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:05.601 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:05.832 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:39 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:05.889 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-cephadm 2024-06-18T12:56:05.957 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:05.958 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:05.958 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-06-18T12:56:05.958 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:05.959 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:56:05.959 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 49 k 2024-06-18T12:56:05.959 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-06-18T12:56:05.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-18T12:56:05.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-18T12:56:05.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:2.30.0-1.el9 epel 213 k 2024-06-18T12:56:05.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-18T12:56:05.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-18T12:56:05.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-18T12:56:05.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-18T12:56:05.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-18T12:56:05.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-18T12:56:05.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-18T12:56:05.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-18T12:56:05.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-18T12:56:05.960 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:05.960 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:56:05.960 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:05.960 INFO:teuthology.orchestra.run.smithi167.stdout:Install 13 Packages 2024-06-18T12:56:05.960 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:05.961 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 2.2 M 2024-06-18T12:56:05.961 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 26 M 2024-06-18T12:56:05.961 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:56:06.458 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:32 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:06.562 INFO:teuthology.orchestra.run.smithi167.stdout:(1/13): ceph-mgr-rook-19.0.0-3124.g3781e3a5.el9 242 kB/s | 49 kB 00:00 2024-06-18T12:56:06.585 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:06.586 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:06.586 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-06-18T12:56:06.586 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:06.586 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:56:06.586 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 142 k 2024-06-18T12:56:06.587 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-06-18T12:56:06.587 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-18T12:56:06.587 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-18T12:56:06.587 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-18T12:56:06.587 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-18T12:56:06.587 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-18T12:56:06.587 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:06.587 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:56:06.587 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:06.587 INFO:teuthology.orchestra.run.smithi179.stdout:Install 6 Packages 2024-06-18T12:56:06.587 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:06.588 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 7.0 M 2024-06-18T12:56:06.588 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 33 M 2024-06-18T12:56:06.588 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:56:06.627 INFO:teuthology.orchestra.run.smithi167.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 71 kB/s | 19 kB 00:00 2024-06-18T12:56:06.653 INFO:teuthology.orchestra.run.smithi167.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 87 kB/s | 26 kB 00:00 2024-06-18T12:56:06.986 INFO:teuthology.orchestra.run.smithi167.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 442 kB/s | 159 kB 00:00 2024-06-18T12:56:07.103 INFO:teuthology.orchestra.run.smithi167.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 619 kB/s | 279 kB 00:00 2024-06-18T12:56:07.125 INFO:teuthology.orchestra.run.smithi179.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 922 kB/s | 248 kB 00:00 2024-06-18T12:56:07.128 INFO:teuthology.orchestra.run.smithi167.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 377 kB/s | 54 kB 00:00 2024-06-18T12:56:07.184 INFO:teuthology.orchestra.run.smithi179.stdout:(2/6): ceph-mgr-cephadm-19.0.0-3124.g3781e3a5.e 434 kB/s | 142 kB 00:00 2024-06-18T12:56:07.212 INFO:teuthology.orchestra.run.smithi167.stdout:(7/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 342 kB/s | 222 kB 00:00 2024-06-18T12:56:07.229 INFO:teuthology.orchestra.run.smithi167.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 256 kB/s | 32 kB 00:00 2024-06-18T12:56:07.243 INFO:teuthology.orchestra.run.smithi179.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.6 MB/s | 548 kB 00:00 2024-06-18T12:56:07.268 INFO:teuthology.orchestra.run.smithi179.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 2.2 MB/s | 51 kB 00:00 2024-06-18T12:56:07.279 INFO:teuthology.orchestra.run.smithi167.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 99 kB/s | 15 kB 00:00 2024-06-18T12:56:07.293 INFO:teuthology.orchestra.run.smithi179.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 531 kB/s | 58 kB 00:00 2024-06-18T12:56:07.313 INFO:teuthology.orchestra.run.smithi167.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.9 MB/s | 59 kB 00:00 2024-06-18T12:56:07.338 INFO:teuthology.orchestra.run.smithi167.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.5 MB/s | 90 kB 00:00 2024-06-18T12:56:07.363 INFO:teuthology.orchestra.run.smithi167.stdout:(12/13): python3-google-auth-2.30.0-1.el9.noarc 1.4 MB/s | 213 kB 00:00 2024-06-18T12:56:07.397 INFO:teuthology.orchestra.run.smithi167.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.1 MB/s | 1.0 MB 00:00 2024-06-18T12:56:07.400 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:07.400 INFO:teuthology.orchestra.run.smithi167.stdout:Total 1.5 MB/s | 2.2 MB 00:01 2024-06-18T12:56:07.466 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:56:07.481 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:56:07.481 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:56:07.571 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:56:07.571 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:56:08.076 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:56:08.260 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-06-18T12:56:08.324 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-06-18T12:56:08.379 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-06-18T12:56:08.416 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-06-18T12:56:08.458 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-06-18T12:56:08.555 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-06-18T12:56:08.669 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:2.30.0-1.el9.noarch 7/13 2024-06-18T12:56:08.718 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-06-18T12:56:08.827 INFO:teuthology.orchestra.run.smithi179.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.0 MB/s | 6.0 MB 00:01 2024-06-18T12:56:08.830 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:08.830 INFO:teuthology.orchestra.run.smithi179.stdout:Total 3.1 MB/s | 7.0 MB 00:02 2024-06-18T12:56:08.902 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:56:08.914 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:56:08.914 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:56:09.012 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:56:09.012 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:56:09.319 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:56:09.436 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-06-18T12:56:09.475 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-06-18T12:56:09.491 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-06-18T12:56:09.520 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-06-18T12:56:09.525 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-06-18T12:56:09.573 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-06-18T12:56:09.603 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:19.0.0-3124.g3781e3a5.el9.noarch 13/13 2024-06-18T12:56:10.033 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-06-18T12:56:10.123 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-06-18T12:56:10.186 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-06-18T12:56:10.216 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 6/6 2024-06-18T12:56:10.430 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3124.g3781e3a5.el9.noarch 13/13 2024-06-18T12:56:10.430 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3124.g3781e3a5.el9.noarch 1/13 2024-06-18T12:56:10.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-06-18T12:56:10.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-06-18T12:56:10.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-06-18T12:56:10.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-06-18T12:56:10.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-06-18T12:56:10.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-06-18T12:56:10.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-06-18T12:56:10.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-06-18T12:56:10.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:2.30.0-1.el9.noarch 10/13 2024-06-18T12:56:10.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-06-18T12:56:10.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-06-18T12:56:10.756 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-06-18T12:56:10.756 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:10.756 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:56:10.756 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:56:10.756 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-18T12:56:10.756 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-18T12:56:10.756 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:2.30.0-1.el9.noarch 2024-06-18T12:56:10.756 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-18T12:56:10.756 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-18T12:56:10.756 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-18T12:56:10.756 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-18T12:56:10.756 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-18T12:56:10.757 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-18T12:56:10.757 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-18T12:56:10.757 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-18T12:56:10.757 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-18T12:56:10.757 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:10.757 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:10.971 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 6/6 2024-06-18T12:56:10.972 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 1/6 2024-06-18T12:56:10.972 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-06-18T12:56:10.972 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-06-18T12:56:10.972 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-06-18T12:56:10.972 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-06-18T12:56:10.997 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr-cephadm 2024-06-18T12:56:11.239 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-06-18T12:56:11.239 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:11.239 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:56:11.239 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:56:11.240 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-18T12:56:11.240 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-18T12:56:11.240 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-18T12:56:11.240 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-18T12:56:11.240 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-18T12:56:11.240 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:11.240 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:11.510 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-fuse 2024-06-18T12:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:45 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:11.693 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:11.693 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:11.694 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-06-18T12:56:11.694 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:11.694 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:56:11.694 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 142 k 2024-06-18T12:56:11.694 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-06-18T12:56:11.694 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-18T12:56:11.695 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-18T12:56:11.695 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-18T12:56:11.695 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-18T12:56:11.695 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-18T12:56:11.695 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:11.695 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:56:11.696 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:11.696 INFO:teuthology.orchestra.run.smithi167.stdout:Install 6 Packages 2024-06-18T12:56:11.696 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:11.697 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 7.0 M 2024-06-18T12:56:11.697 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 33 M 2024-06-18T12:56:11.697 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:56:12.084 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:38 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:12.093 INFO:teuthology.orchestra.run.smithi167.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-06-18T12:56:12.118 INFO:teuthology.orchestra.run.smithi167.stdout:(2/6): ceph-mgr-cephadm-19.0.0-3124.g3781e3a5.e 577 kB/s | 142 kB 00:00 2024-06-18T12:56:12.214 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:12.215 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-06-18T12:56:12.215 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:12.215 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:56:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 893 k 2024-06-18T12:56:12.215 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-06-18T12:56:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-18T12:56:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:12.215 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:56:12.215 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:12.215 INFO:teuthology.orchestra.run.smithi179.stdout:Install 2 Packages 2024-06-18T12:56:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:12.216 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 973 k 2024-06-18T12:56:12.216 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 2.8 M 2024-06-18T12:56:12.216 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:56:12.218 INFO:teuthology.orchestra.run.smithi167.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.3 MB/s | 548 kB 00:00 2024-06-18T12:56:12.244 INFO:teuthology.orchestra.run.smithi167.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 460 kB/s | 58 kB 00:00 2024-06-18T12:56:12.269 INFO:teuthology.orchestra.run.smithi167.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 2.0 MB/s | 51 kB 00:00 2024-06-18T12:56:12.395 INFO:teuthology.orchestra.run.smithi179.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 587 kB/s | 80 kB 00:00 2024-06-18T12:56:12.661 INFO:teuthology.orchestra.run.smithi167.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 7.6 MB/s | 6.0 MB 00:00 2024-06-18T12:56:12.663 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:12.664 INFO:teuthology.orchestra.run.smithi167.stdout:Total 7.2 MB/s | 7.0 MB 00:00 2024-06-18T12:56:12.687 INFO:teuthology.orchestra.run.smithi179.stdout:(2/2): ceph-fuse-19.0.0-3124.g3781e3a5.el9.x86_ 2.0 MB/s | 893 kB 00:00 2024-06-18T12:56:12.688 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:12.688 INFO:teuthology.orchestra.run.smithi179.stdout:Total 2.0 MB/s | 973 kB 00:00 2024-06-18T12:56:12.699 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:56:12.722 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:56:12.722 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:56:12.733 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:56:12.746 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:56:12.746 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:56:12.801 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:56:12.801 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:56:12.843 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:56:12.844 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:56:13.006 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:56:13.139 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-06-18T12:56:13.155 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:56:13.212 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 2/2 2024-06-18T12:56:13.327 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-06-18T12:56:13.388 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-06-18T12:56:13.903 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-06-18T12:56:13.916 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 2/2 2024-06-18T12:56:13.916 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/2 2024-06-18T12:56:13.991 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-06-18T12:56:14.054 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-06-18T12:56:14.084 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 6/6 2024-06-18T12:56:14.217 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-06-18T12:56:14.217 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:14.217 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:56:14.218 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-06-18T12:56:14.218 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:14.218 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:14.485 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-volume 2024-06-18T12:56:14.856 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 6/6 2024-06-18T12:56:14.856 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 1/6 2024-06-18T12:56:14.857 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-06-18T12:56:14.857 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-06-18T12:56:14.857 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-06-18T12:56:14.857 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-06-18T12:56:15.073 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:41 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:15.129 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-06-18T12:56:15.129 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:15.130 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:56:15.130 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:56:15.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-18T12:56:15.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-18T12:56:15.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-18T12:56:15.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-18T12:56:15.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-18T12:56:15.130 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:15.130 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:15.204 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:15.205 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:15.205 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-06-18T12:56:15.205 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:15.205 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:56:15.205 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 264 k 2024-06-18T12:56:15.205 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-06-18T12:56:15.205 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-18T12:56:15.205 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:15.205 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:56:15.205 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:15.206 INFO:teuthology.orchestra.run.smithi179.stdout:Install 2 Packages 2024-06-18T12:56:15.206 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:15.206 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 341 k 2024-06-18T12:56:15.206 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 1.5 M 2024-06-18T12:56:15.206 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:56:15.363 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-fuse 2024-06-18T12:56:15.601 INFO:teuthology.orchestra.run.smithi179.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 642 kB/s | 77 kB 00:00 2024-06-18T12:56:15.759 INFO:teuthology.orchestra.run.smithi179.stdout:(2/2): ceph-volume-19.0.0-3124.g3781e3a5.el9.no 946 kB/s | 264 kB 00:00 2024-06-18T12:56:15.759 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:15.760 INFO:teuthology.orchestra.run.smithi179.stdout:Total 615 kB/s | 341 kB 00:00 2024-06-18T12:56:15.771 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:56:15.781 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:56:15.782 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:56:15.817 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:56:15.818 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:56:15.945 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:49 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:15.960 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:56:16.072 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:16.073 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:16.073 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-06-18T12:56:16.073 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:16.073 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:56:16.073 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 893 k 2024-06-18T12:56:16.073 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-06-18T12:56:16.073 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-18T12:56:16.073 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:16.073 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:56:16.073 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:16.073 INFO:teuthology.orchestra.run.smithi167.stdout:Install 2 Packages 2024-06-18T12:56:16.073 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:16.074 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 973 k 2024-06-18T12:56:16.074 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 2.8 M 2024-06-18T12:56:16.074 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:56:16.142 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-06-18T12:56:16.178 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-volume-2:19.0.0-3124.g3781e3a5.el9.noarch 2/2 2024-06-18T12:56:16.194 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-volume-2:19.0.0-3124.g3781e3a5.el9.noarch 2/2 2024-06-18T12:56:16.194 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:56:16.194 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-18T12:56:16.195 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:16.431 INFO:teuthology.orchestra.run.smithi167.stdout:(1/2): ceph-fuse-19.0.0-3124.g3781e3a5.el9.x86_ 3.0 MB/s | 893 kB 00:00 2024-06-18T12:56:16.573 INFO:teuthology.orchestra.run.smithi167.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 188 kB/s | 80 kB 00:00 2024-06-18T12:56:16.573 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:16.573 INFO:teuthology.orchestra.run.smithi167.stdout:Total 1.9 MB/s | 973 kB 00:00 2024-06-18T12:56:16.585 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:56:16.607 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:56:16.607 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:56:16.685 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:56:16.685 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:56:16.882 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:56:16.994 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-volume-2:19.0.0-3124.g3781e3a5.el9.noarch 1/2 2024-06-18T12:56:17.008 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-06-18T12:56:17.072 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 2/2 2024-06-18T12:56:17.298 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-06-18T12:56:17.299 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:17.299 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:56:17.299 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:56:17.299 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-18T12:56:17.299 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:17.299 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:17.616 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install librados-devel 2024-06-18T12:56:17.707 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 2/2 2024-06-18T12:56:17.708 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/2 2024-06-18T12:56:17.981 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-06-18T12:56:17.981 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:17.981 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:56:17.981 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-06-18T12:56:17.982 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:17.982 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:18.194 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:44 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:18.239 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-volume 2024-06-18T12:56:18.322 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:18.323 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:18.323 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repo Size 2024-06-18T12:56:18.323 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:18.323 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:56:18.324 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 127 k 2024-06-18T12:56:18.324 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:18.324 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:56:18.324 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:18.324 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-06-18T12:56:18.324 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:18.324 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 127 k 2024-06-18T12:56:18.324 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 456 k 2024-06-18T12:56:18.324 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:56:18.548 INFO:teuthology.orchestra.run.smithi179.stdout:librados-devel-19.0.0-3124.g3781e3a5.el9.x86_64 567 kB/s | 127 kB 00:00 2024-06-18T12:56:18.549 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:18.549 INFO:teuthology.orchestra.run.smithi179.stdout:Total 562 kB/s | 127 kB 00:00 2024-06-18T12:56:18.549 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:56:18.554 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:56:18.554 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:56:18.617 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:56:18.618 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:56:18.771 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:56:18.817 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:52 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:18.870 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librados-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:18.948 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:18.948 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:18.948 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-06-18T12:56:18.949 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:18.949 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:56:18.949 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:19.0.0-3124.g3781e3a5.el9 ceph-noarch 264 k 2024-06-18T12:56:18.949 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-06-18T12:56:18.949 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-18T12:56:18.949 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:18.949 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:56:18.949 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:18.949 INFO:teuthology.orchestra.run.smithi167.stdout:Install 2 Packages 2024-06-18T12:56:18.949 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:18.950 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 341 k 2024-06-18T12:56:18.950 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 1.5 M 2024-06-18T12:56:18.950 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:56:19.229 INFO:teuthology.orchestra.run.smithi167.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 522 kB/s | 77 kB 00:00 2024-06-18T12:56:19.313 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:19.438 INFO:teuthology.orchestra.run.smithi167.stdout:(2/2): ceph-volume-19.0.0-3124.g3781e3a5.el9.no 740 kB/s | 264 kB 00:00 2024-06-18T12:56:19.438 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:19.439 INFO:teuthology.orchestra.run.smithi167.stdout:Total 697 kB/s | 341 kB 00:00 2024-06-18T12:56:19.450 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:56:19.460 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:56:19.461 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:56:19.496 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:56:19.496 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:56:19.580 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:19.581 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:19.581 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:56:19.581 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:56:19.581 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:19.581 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:19.640 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:56:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-06-18T12:56:19.855 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install libcephfs2 2024-06-18T12:56:19.865 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:19.0.0-3124.g3781e3a5.el9.noarch 2/2 2024-06-18T12:56:19.881 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:19.0.0-3124.g3781e3a5.el9.noarch 2/2 2024-06-18T12:56:19.881 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:56:19.881 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-18T12:56:19.881 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:20.441 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:46 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:20.526 INFO:teuthology.orchestra.run.smithi179.stdout:Package libcephfs2-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:56:20.550 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:19.0.0-3124.g3781e3a5.el9.noarch 1/2 2024-06-18T12:56:20.570 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:20.571 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-06-18T12:56:20.571 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:20.642 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install libcephfs-devel 2024-06-18T12:56:20.810 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-06-18T12:56:20.810 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:20.810 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:56:20.810 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:19.0.0-3124.g3781e3a5.el9.noarch 2024-06-18T12:56:20.810 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-18T12:56:20.810 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:20.810 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:21.137 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install librados-devel 2024-06-18T12:56:21.212 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:47 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:21.337 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:21.338 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:21.338 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repo Size 2024-06-18T12:56:21.338 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:21.339 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:56:21.339 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 32 k 2024-06-18T12:56:21.339 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:21.339 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:56:21.339 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:21.339 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-06-18T12:56:21.339 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:21.339 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 32 k 2024-06-18T12:56:21.339 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 142 k 2024-06-18T12:56:21.339 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:56:21.585 INFO:teuthology.orchestra.run.smithi179.stdout:libcephfs-devel-19.0.0-3124.g3781e3a5.el9.x86_6 129 kB/s | 32 kB 00:00 2024-06-18T12:56:21.586 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:21.586 INFO:teuthology.orchestra.run.smithi179.stdout:Total 128 kB/s | 32 kB 00:00 2024-06-18T12:56:21.586 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:56:21.589 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:56:21.590 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:56:21.610 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:56:21.610 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:56:21.687 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:56:21.719 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:55 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:21.800 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:21.845 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:21.846 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:21.846 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repo Size 2024-06-18T12:56:21.846 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:21.846 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:56:21.846 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 127 k 2024-06-18T12:56:21.846 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:21.846 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:56:21.846 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:21.846 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2024-06-18T12:56:21.846 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:21.847 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 127 k 2024-06-18T12:56:21.847 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 456 k 2024-06-18T12:56:21.847 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:56:22.162 INFO:teuthology.orchestra.run.smithi167.stdout:librados-devel-19.0.0-3124.g3781e3a5.el9.x86_64 402 kB/s | 127 kB 00:00 2024-06-18T12:56:22.163 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:22.163 INFO:teuthology.orchestra.run.smithi167.stdout:Total 400 kB/s | 127 kB 00:00 2024-06-18T12:56:22.163 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:56:22.168 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:56:22.169 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:56:22.188 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:22.231 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:56:22.231 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:56:22.378 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:56:22.424 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:22.424 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:22.424 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:56:22.424 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:56:22.424 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:22.424 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:22.494 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:22.597 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install librados2 2024-06-18T12:56:22.965 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:23.167 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:49 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:23.221 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:23.221 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:23.221 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:56:23.221 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:56:23.221 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:23.221 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:23.252 INFO:teuthology.orchestra.run.smithi179.stdout:Package librados2-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:56:23.294 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:23.295 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-06-18T12:56:23.295 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:23.366 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install librbd1 2024-06-18T12:56:23.492 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install libcephfs2 2024-06-18T12:56:23.937 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:49 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:24.021 INFO:teuthology.orchestra.run.smithi179.stdout:Package librbd1-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:56:24.062 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:24.063 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-06-18T12:56:24.063 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:24.082 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:58 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:24.144 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-rados 2024-06-18T12:56:24.170 INFO:teuthology.orchestra.run.smithi167.stdout:Package libcephfs2-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:56:24.212 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:24.214 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2024-06-18T12:56:24.214 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:24.287 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install libcephfs-devel 2024-06-18T12:56:24.715 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:50 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:24.798 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-rados-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:56:24.840 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:24.841 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-06-18T12:56:24.841 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:24.856 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:58 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:24.907 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-rgw 2024-06-18T12:56:24.982 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:24.982 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:24.982 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repo Size 2024-06-18T12:56:24.983 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:24.983 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:56:24.983 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 32 k 2024-06-18T12:56:24.983 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:24.983 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:56:24.983 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:24.983 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2024-06-18T12:56:24.983 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:24.983 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 32 k 2024-06-18T12:56:24.983 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 142 k 2024-06-18T12:56:24.983 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:56:25.239 INFO:teuthology.orchestra.run.smithi167.stdout:libcephfs-devel-19.0.0-3124.g3781e3a5.el9.x86_6 124 kB/s | 32 kB 00:00 2024-06-18T12:56:25.240 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:25.240 INFO:teuthology.orchestra.run.smithi167.stdout:Total 123 kB/s | 32 kB 00:00 2024-06-18T12:56:25.240 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:56:25.243 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:56:25.243 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:56:25.264 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:56:25.265 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:56:25.344 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:56:25.446 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:25.477 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:51 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:25.561 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-rgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:56:25.603 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:25.603 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-06-18T12:56:25.604 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:25.686 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-cephfs 2024-06-18T12:56:25.832 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:26.045 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:26.045 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:26.045 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:56:26.045 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:56:26.045 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:26.045 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:26.223 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install librados2 2024-06-18T12:56:26.258 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:52 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:26.343 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-cephfs-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:56:26.385 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:26.386 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-06-18T12:56:26.386 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:26.457 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-rbd 2024-06-18T12:56:26.797 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:00 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:26.881 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:56:26.924 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:26.924 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2024-06-18T12:56:26.925 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:26.987 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install librbd1 2024-06-18T12:56:27.030 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:53 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:27.114 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-rbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:56:27.156 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:27.157 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-06-18T12:56:27.158 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:27.224 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install rbd-fuse 2024-06-18T12:56:27.561 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:01 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:27.647 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:56:27.689 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:27.691 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2024-06-18T12:56:27.691 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:27.750 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install python3-rados 2024-06-18T12:56:27.805 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:53 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:27.940 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:27.940 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:27.940 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-06-18T12:56:27.941 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:27.941 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:56:27.941 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 85 k 2024-06-18T12:56:27.941 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:27.941 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:56:27.941 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:27.941 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-06-18T12:56:27.941 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:27.941 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 85 k 2024-06-18T12:56:27.941 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 231 k 2024-06-18T12:56:27.941 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:56:28.151 INFO:teuthology.orchestra.run.smithi179.stdout:rbd-fuse-19.0.0-3124.g3781e3a5.el9.x86_64.rpm 408 kB/s | 85 kB 00:00 2024-06-18T12:56:28.151 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:28.152 INFO:teuthology.orchestra.run.smithi179.stdout:Total 404 kB/s | 85 kB 00:00 2024-06-18T12:56:28.152 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:56:28.159 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:56:28.159 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:56:28.225 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:56:28.225 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:56:28.320 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:02 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:28.374 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:56:28.405 INFO:teuthology.orchestra.run.smithi167.stdout:Package python3-rados-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:56:28.448 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:28.449 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2024-06-18T12:56:28.449 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:28.476 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:28.511 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install python3-rgw 2024-06-18T12:56:28.883 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:29.087 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:03 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:29.150 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:29.150 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:29.150 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:56:29.150 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:56:29.150 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:29.150 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:29.170 INFO:teuthology.orchestra.run.smithi167.stdout:Package python3-rgw-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:56:29.212 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:29.213 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2024-06-18T12:56:29.213 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:29.279 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install python3-cephfs 2024-06-18T12:56:29.370 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install rbd-mirror 2024-06-18T12:56:29.852 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:03 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:29.936 INFO:teuthology.orchestra.run.smithi167.stdout:Package python3-cephfs-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:56:29.949 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:55 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:29.978 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:29.979 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2024-06-18T12:56:29.979 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:30.041 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install python3-rbd 2024-06-18T12:56:30.077 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:30.078 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:30.078 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repo Size 2024-06-18T12:56:30.078 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:30.078 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:56:30.078 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 3.1 M 2024-06-18T12:56:30.078 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:30.078 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:56:30.078 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:30.078 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-06-18T12:56:30.079 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:30.079 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 3.1 M 2024-06-18T12:56:30.079 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 13 M 2024-06-18T12:56:30.079 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:56:30.614 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:04 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:30.697 INFO:teuthology.orchestra.run.smithi167.stdout:Package python3-rbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 is already installed. 2024-06-18T12:56:30.707 INFO:teuthology.orchestra.run.smithi179.stdout:rbd-mirror-19.0.0-3124.g3781e3a5.el9.x86_64.rpm 4.9 MB/s | 3.1 MB 00:00 2024-06-18T12:56:30.708 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:30.709 INFO:teuthology.orchestra.run.smithi179.stdout:Total 4.9 MB/s | 3.1 MB 00:00 2024-06-18T12:56:30.709 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:56:30.717 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:56:30.717 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:56:30.739 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:30.740 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2024-06-18T12:56:30.740 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:30.798 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:56:30.799 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:56:30.801 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install rbd-fuse 2024-06-18T12:56:31.224 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:56:31.284 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-mirror-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:31.310 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:31.310 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:56:31.310 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-18T12:56:31.310 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-18T12:56:31.310 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-18T12:56:31.310 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:31.383 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:05 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:31.518 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:31.519 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:31.519 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2024-06-18T12:56:31.519 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:31.519 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:56:31.519 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 85 k 2024-06-18T12:56:31.519 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:31.519 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:56:31.520 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:31.520 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2024-06-18T12:56:31.520 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:31.520 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 85 k 2024-06-18T12:56:31.520 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 231 k 2024-06-18T12:56:31.520 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:56:31.836 INFO:teuthology.orchestra.run.smithi167.stdout:rbd-fuse-19.0.0-3124.g3781e3a5.el9.x86_64.rpm 269 kB/s | 85 kB 00:00 2024-06-18T12:56:31.837 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:31.837 INFO:teuthology.orchestra.run.smithi167.stdout:Total 268 kB/s | 85 kB 00:00 2024-06-18T12:56:31.837 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:56:31.845 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:56:31.845 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:56:31.908 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:56:31.909 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:56:32.077 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:56:32.193 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:32.358 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-mirror-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:32.358 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:32.358 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:56:32.358 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:56:32.358 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:32.358 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:32.582 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install rbd-nbd 2024-06-18T12:56:32.591 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:32.869 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:32.869 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:32.869 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:56:32.869 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:56:32.869 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:32.870 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:33.130 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install rbd-mirror 2024-06-18T12:56:33.169 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:59 ago on Tue 18 Jun 2024 12:54:34 PM UTC. 2024-06-18T12:56:33.300 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-18T12:56:33.300 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:33.301 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-06-18T12:56:33.301 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:33.301 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-18T12:56:33.301 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 171 k 2024-06-18T12:56:33.301 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:33.301 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-18T12:56:33.301 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-06-18T12:56:33.301 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-06-18T12:56:33.301 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:33.301 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 171 k 2024-06-18T12:56:33.301 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 494 k 2024-06-18T12:56:33.302 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-18T12:56:33.533 INFO:teuthology.orchestra.run.smithi179.stdout:rbd-nbd-19.0.0-3124.g3781e3a5.el9.x86_64.rpm 738 kB/s | 171 kB 00:00 2024-06-18T12:56:33.534 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:33.534 INFO:teuthology.orchestra.run.smithi179.stdout:Total 733 kB/s | 171 kB 00:00 2024-06-18T12:56:33.534 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-18T12:56:33.542 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-18T12:56:33.542 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-18T12:56:33.605 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-18T12:56:33.606 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-18T12:56:33.719 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:07 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:33.757 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-18T12:56:33.851 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:33.852 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:33.852 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repo Size 2024-06-18T12:56:33.852 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:33.852 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:56:33.852 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 3.1 M 2024-06-18T12:56:33.852 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:33.852 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:56:33.852 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:33.853 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2024-06-18T12:56:33.853 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:33.853 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 3.1 M 2024-06-18T12:56:33.853 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 13 M 2024-06-18T12:56:33.853 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:56:33.865 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-nbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:34.282 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:34.536 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-nbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:34.536 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:34.536 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-18T12:56:34.536 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:56:34.536 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:56:34.536 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-18T12:56:34.559 INFO:teuthology.orchestra.run.smithi167.stdout:rbd-mirror-19.0.0-3124.g3781e3a5.el9.x86_64.rpm 4.4 MB/s | 3.1 MB 00:00 2024-06-18T12:56:34.560 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:34.561 INFO:teuthology.orchestra.run.smithi167.stdout:Total 4.4 MB/s | 3.1 MB 00:00 2024-06-18T12:56:34.561 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:56:34.570 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:56:34.570 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:56:34.649 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:56:34.650 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:56:34.800 DEBUG:teuthology.parallel:result is None 2024-06-18T12:56:35.071 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:56:35.136 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:35.161 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:35.162 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-18T12:56:35.162 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-18T12:56:35.162 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-18T12:56:35.162 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-18T12:56:35.162 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:36.260 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:36.261 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:36.261 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:56:36.261 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:56:36.261 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:36.261 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:36.455 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install rbd-nbd 2024-06-18T12:56:37.040 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:11 ago on Tue 18 Jun 2024 12:54:26 PM UTC. 2024-06-18T12:56:37.172 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-18T12:56:37.173 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:37.173 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2024-06-18T12:56:37.173 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:37.173 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-18T12:56:37.173 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:19.0.0-3124.g3781e3a5.el9 ceph 171 k 2024-06-18T12:56:37.173 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:37.174 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-18T12:56:37.174 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-06-18T12:56:37.174 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2024-06-18T12:56:37.174 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:37.174 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 171 k 2024-06-18T12:56:37.174 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 494 k 2024-06-18T12:56:37.174 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-18T12:56:37.400 INFO:teuthology.orchestra.run.smithi167.stdout:rbd-nbd-19.0.0-3124.g3781e3a5.el9.x86_64.rpm 760 kB/s | 171 kB 00:00 2024-06-18T12:56:37.400 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:56:37.401 INFO:teuthology.orchestra.run.smithi167.stdout:Total 754 kB/s | 171 kB 00:00 2024-06-18T12:56:37.401 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-18T12:56:37.408 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-18T12:56:37.408 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-18T12:56:37.473 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-18T12:56:37.473 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-18T12:56:37.633 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-18T12:56:37.748 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:38.193 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:38.435 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 1/1 2024-06-18T12:56:38.436 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:38.436 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-18T12:56:38.436 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:19.0.0-3124.g3781e3a5.el9.x86_64 2024-06-18T12:56:38.436 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:56:38.436 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-18T12:56:38.697 DEBUG:teuthology.parallel:result is None 2024-06-18T12:56:38.698 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3781e3a592387d16e8020f5db99cdee4198f6d00 2024-06-18T12:56:38.838 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-18T12:56:38.859 INFO:teuthology.orchestra.run.smithi167.stdout:19.0.0-3124.g3781e3a5.el9 2024-06-18T12:56:38.860 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3124.g3781e3a5.el9 2024-06-18T12:56:38.860 INFO:teuthology.task.install:The correct ceph version 19.0.0-3124.g3781e3a5 is installed. 2024-06-18T12:56:38.862 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3781e3a592387d16e8020f5db99cdee4198f6d00 2024-06-18T12:56:38.993 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-18T12:56:39.016 INFO:teuthology.orchestra.run.smithi179.stdout:19.0.0-3124.g3781e3a5.el9 2024-06-18T12:56:39.016 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3124.g3781e3a5.el9 2024-06-18T12:56:39.017 INFO:teuthology.task.install:The correct ceph version 19.0.0-3124.g3781e3a5 is installed. 2024-06-18T12:56:39.018 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-18T12:56:39.019 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T12:56:39.019 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-18T12:56:39.044 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-18T12:56:39.044 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-18T12:56:39.083 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-18T12:56:39.083 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T12:56:39.083 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-06-18T12:56:39.109 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-18T12:56:39.173 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-18T12:56:39.173 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/daemon-helper 2024-06-18T12:56:39.199 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-18T12:56:39.264 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-18T12:56:39.265 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T12:56:39.265 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-18T12:56:39.289 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-18T12:56:39.354 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-18T12:56:39.354 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-18T12:56:39.379 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-18T12:56:39.444 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-18T12:56:39.444 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T12:56:39.444 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-06-18T12:56:39.469 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-18T12:56:39.534 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-18T12:56:39.534 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/stdin-killer 2024-06-18T12:56:39.558 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-18T12:56:39.623 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-18T12:56:39.717 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'log-only-match': ['CEPHADM_'], 'sha1': '3781e3a592387d16e8020f5db99cdee4198f6d00', 'cephadm_mode': 'root'} 2024-06-18T12:56:39.717 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 2024-06-18T12:56:39.718 INFO:tasks.cephadm:Cluster fsid is 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:56:39.718 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-18T12:56:39.718 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.167', 'mon.c': '[v2:172.21.15.167:3301,v1:172.21.15.167:6790]', 'mon.b': '172.21.15.179'} 2024-06-18T12:56:39.718 INFO:tasks.cephadm:First mon is mon.a on smithi167 2024-06-18T12:56:39.718 INFO:tasks.cephadm:First mgr is y 2024-06-18T12:56:39.718 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-18T12:56:39.719 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname $(hostname -s) 2024-06-18T12:56:39.745 DEBUG:teuthology.orchestra.run.smithi179:> sudo hostname $(hostname -s) 2024-06-18T12:56:39.770 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-18T12:56:39.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3781e3a592387d16e8020f5db99cdee4198f6d00 2024-06-18T12:56:39.905 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3781e3a592387d16e8020f5db99cdee4198f6d00', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55401/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3124-g3781e3a5', 'node_name': '172.21.5.32+adami02', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3124.g3781e3a5'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/3781e3a592387d16e8020f5db99cdee4198f6d00/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-06-17 16:23:30.607990', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/3781e3a592387d16e8020f5db99cdee4198f6d00/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-06-18T12:56:40.123 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 3781e3a592387d16e8020f5db99cdee4198f6d00 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3781e3a592387d16e8020f5db99cdee4198f6d00 2024-06-18T12:56:40.125 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/3781e3a592387d16e8020f5db99cdee4198f6d00/centos/9/x86_64/flavors/default/cephadm 2024-06-18T12:56:40.125 DEBUG:teuthology.orchestra.run.smithi167:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/3781e3a592387d16e8020f5db99cdee4198f6d00/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-18T12:56:40.693 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-r--r--. 1 ubuntu ubuntu 781806 Jun 18 12:56 /home/ubuntu/cephtest/cephadm 2024-06-18T12:56:40.693 DEBUG:teuthology.orchestra.run.smithi179:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/3781e3a592387d16e8020f5db99cdee4198f6d00/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-18T12:56:41.264 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r--. 1 ubuntu ubuntu 781806 Jun 18 12:56 /home/ubuntu/cephtest/cephadm 2024-06-18T12:56:41.264 DEBUG:teuthology.orchestra.run.smithi167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-18T12:56:41.284 DEBUG:teuthology.orchestra.run.smithi179:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-18T12:56:41.320 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 on all hosts... 2024-06-18T12:56:41.320 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 pull 2024-06-18T12:56:41.327 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 pull 2024-06-18T12:56:42.225 INFO:teuthology.orchestra.run.smithi179.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00... 2024-06-18T12:56:42.303 INFO:teuthology.orchestra.run.smithi167.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00... 2024-06-18T12:57:09.099 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-06-18T12:57:09.100 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 19.0.0-3124-g3781e3a5 (3781e3a592387d16e8020f5db99cdee4198f6d00) squid (rc)", 2024-06-18T12:57:09.100 INFO:teuthology.orchestra.run.smithi179.stdout: "image_id": "af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44", 2024-06-18T12:57:09.100 INFO:teuthology.orchestra.run.smithi179.stdout: "repo_digests": [ 2024-06-18T12:57:09.100 INFO:teuthology.orchestra.run.smithi179.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314" 2024-06-18T12:57:09.100 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-06-18T12:57:09.100 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-06-18T12:57:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-06-18T12:57:10.224 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.0.0-3124-g3781e3a5 (3781e3a592387d16e8020f5db99cdee4198f6d00) squid (rc)", 2024-06-18T12:57:10.224 INFO:teuthology.orchestra.run.smithi167.stdout: "image_id": "af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44", 2024-06-18T12:57:10.224 INFO:teuthology.orchestra.run.smithi167.stdout: "repo_digests": [ 2024-06-18T12:57:10.224 INFO:teuthology.orchestra.run.smithi167.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314" 2024-06-18T12:57:10.224 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-06-18T12:57:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-06-18T12:57:10.249 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph 2024-06-18T12:57:10.291 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph 2024-06-18T12:57:10.324 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /etc/ceph 2024-06-18T12:57:10.355 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 777 /etc/ceph 2024-06-18T12:57:10.392 INFO:tasks.cephadm:Writing seed config... 2024-06-18T12:57:10.393 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-18T12:57:10.393 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-18T12:57:10.393 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-18T12:57:10.393 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-06-18T12:57:10.393 INFO:tasks.cephadm: override: [global] ms type = async 2024-06-18T12:57:10.393 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-18T12:57:10.393 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-18T12:57:10.393 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-18T12:57:10.393 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-06-18T12:57:10.393 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-18T12:57:10.393 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-18T12:57:10.394 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-06-18T12:57:10.394 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-06-18T12:57:10.394 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-18T12:57:10.394 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T12:57:10.394 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-18T12:57:10.410 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3434f1e2-2d72-11ef-bc9d-c7b262605968 mon election default strategy = 1 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-06-18T12:57:10.411 DEBUG:teuthology.orchestra.run.smithi167:mon.a> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.a.service 2024-06-18T12:57:10.453 DEBUG:teuthology.orchestra.run.smithi167:mgr.y> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mgr.y.service 2024-06-18T12:57:10.496 INFO:tasks.cephadm:Bootstrapping... 2024-06-18T12:57:10.496 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 -v bootstrap --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.167 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-18T12:57:10.688 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-18T12:57:10.688 INFO:teuthology.orchestra.run.smithi167.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00', '-v', 'bootstrap', '--fsid', '3434f1e2-2d72-11ef-bc9d-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.167', '--skip-admin-label'] 2024-06-18T12:57:10.688 INFO:teuthology.orchestra.run.smithi167.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-18T12:57:10.689 INFO:teuthology.orchestra.run.smithi167.stdout:Verifying podman|docker is present... 2024-06-18T12:57:10.713 INFO:teuthology.orchestra.run.smithi167.stdout:/bin/podman: stdout 5.1.0 2024-06-18T12:57:10.713 INFO:teuthology.orchestra.run.smithi167.stdout:Verifying lvm2 is present... 2024-06-18T12:57:10.714 INFO:teuthology.orchestra.run.smithi167.stdout:Verifying time synchronization is in place... 2024-06-18T12:57:10.721 INFO:teuthology.orchestra.run.smithi167.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-18T12:57:10.721 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-18T12:57:10.728 INFO:teuthology.orchestra.run.smithi167.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-18T12:57:10.728 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stdout inactive 2024-06-18T12:57:10.734 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stdout enabled 2024-06-18T12:57:10.741 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stdout active 2024-06-18T12:57:10.741 INFO:teuthology.orchestra.run.smithi167.stdout:Unit chronyd.service is enabled and running 2024-06-18T12:57:10.741 INFO:teuthology.orchestra.run.smithi167.stdout:Repeating the final host check... 2024-06-18T12:57:10.764 INFO:teuthology.orchestra.run.smithi167.stdout:/bin/podman: stdout 5.1.0 2024-06-18T12:57:10.764 INFO:teuthology.orchestra.run.smithi167.stdout:podman (/bin/podman) version 5.1.0 is present 2024-06-18T12:57:10.764 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl is present 2024-06-18T12:57:10.764 INFO:teuthology.orchestra.run.smithi167.stdout:lvcreate is present 2024-06-18T12:57:10.770 INFO:teuthology.orchestra.run.smithi167.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-18T12:57:10.771 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-18T12:57:10.777 INFO:teuthology.orchestra.run.smithi167.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-18T12:57:10.777 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stdout inactive 2024-06-18T12:57:10.783 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stdout enabled 2024-06-18T12:57:10.790 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stdout active 2024-06-18T12:57:10.790 INFO:teuthology.orchestra.run.smithi167.stdout:Unit chronyd.service is enabled and running 2024-06-18T12:57:10.790 INFO:teuthology.orchestra.run.smithi167.stdout:Host looks OK 2024-06-18T12:57:10.790 INFO:teuthology.orchestra.run.smithi167.stdout:Cluster fsid: 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:57:10.790 INFO:teuthology.orchestra.run.smithi167.stdout:Acquiring lock 140529246565760 on /run/cephadm/3434f1e2-2d72-11ef-bc9d-c7b262605968.lock 2024-06-18T12:57:10.790 INFO:teuthology.orchestra.run.smithi167.stdout:Lock 140529246565760 acquired on /run/cephadm/3434f1e2-2d72-11ef-bc9d-c7b262605968.lock 2024-06-18T12:57:10.791 INFO:teuthology.orchestra.run.smithi167.stdout:Verifying IP 172.21.15.167 port 3300 ... 2024-06-18T12:57:10.791 INFO:teuthology.orchestra.run.smithi167.stdout:Verifying IP 172.21.15.167 port 6789 ... 2024-06-18T12:57:10.791 INFO:teuthology.orchestra.run.smithi167.stdout:Base mon IP(s) is [172.21.15.167:3300, 172.21.15.167:6789], mon addrv is [v2:172.21.15.167:3300,v1:172.21.15.167:6789] 2024-06-18T12:57:10.794 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.167 metric 100 2024-06-18T12:57:10.795 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.167 metric 100 2024-06-18T12:57:10.797 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-18T12:57:10.797 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-06-18T12:57:10.797 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1784sec hoplimit 64 pref medium 2024-06-18T12:57:10.799 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-18T12:57:10.800 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-18T12:57:10.800 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-18T12:57:10.800 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-06-18T12:57:10.800 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-06-18T12:57:10.800 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-18T12:57:10.800 INFO:teuthology.orchestra.run.smithi167.stdout:Mon IP `172.21.15.167` is in CIDR network `172.21.0.0/20` 2024-06-18T12:57:10.801 INFO:teuthology.orchestra.run.smithi167.stdout:Mon IP `172.21.15.167` is in CIDR network `172.21.0.0/20` 2024-06-18T12:57:10.801 INFO:teuthology.orchestra.run.smithi167.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-18T12:57:10.801 INFO:teuthology.orchestra.run.smithi167.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-18T12:57:10.801 INFO:teuthology.orchestra.run.smithi167.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00... 2024-06-18T12:57:11.276 INFO:teuthology.orchestra.run.smithi167.stdout:/bin/podman: stdout af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44 2024-06-18T12:57:11.276 INFO:teuthology.orchestra.run.smithi167.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00... 2024-06-18T12:57:11.276 INFO:teuthology.orchestra.run.smithi167.stdout:/bin/podman: stderr Getting image source signatures 2024-06-18T12:57:11.276 INFO:teuthology.orchestra.run.smithi167.stdout:/bin/podman: stderr Copying blob sha256:2e26d1010ba3f18af1bbb340ee0d21697ea3adcaa248a1e6e2a1168cffe78001 2024-06-18T12:57:11.276 INFO:teuthology.orchestra.run.smithi167.stdout:/bin/podman: stderr Copying blob sha256:9065cd298533077f0006921bb12bd331d24f6c140cc8fa972c2e6c4e382038d1 2024-06-18T12:57:11.277 INFO:teuthology.orchestra.run.smithi167.stdout:/bin/podman: stderr Copying config sha256:af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44 2024-06-18T12:57:11.277 INFO:teuthology.orchestra.run.smithi167.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-18T12:57:12.150 INFO:teuthology.orchestra.run.smithi167.stdout:ceph: stdout ceph version 19.0.0-3124-g3781e3a5 (3781e3a592387d16e8020f5db99cdee4198f6d00) squid (rc) 2024-06-18T12:57:12.150 INFO:teuthology.orchestra.run.smithi167.stdout:Ceph version: ceph version 19.0.0-3124-g3781e3a5 (3781e3a592387d16e8020f5db99cdee4198f6d00) squid (rc) 2024-06-18T12:57:12.150 INFO:teuthology.orchestra.run.smithi167.stdout:Extracting ceph user uid/gid from container image... 2024-06-18T12:57:12.964 INFO:teuthology.orchestra.run.smithi167.stdout:stat: stdout 167 167 2024-06-18T12:57:12.964 INFO:teuthology.orchestra.run.smithi167.stdout:Creating initial keys... 2024-06-18T12:57:13.853 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph-authtool: stdout AQAphHFmZVLKDhAA/syafpHBwV6xxizYHRZRfA== 2024-06-18T12:57:14.759 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph-authtool: stdout AQAqhHFmi3nDCRAAeN3MY5l/LUb5Nw8qSy3ZRg== 2024-06-18T12:57:15.665 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph-authtool: stdout AQArhHFmIxMMCBAA4eKYqSPzgghaELlABg7eUQ== 2024-06-18T12:57:15.666 INFO:teuthology.orchestra.run.smithi167.stdout:Creating initial monmap... 2024-06-18T12:57:16.563 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-18T12:57:16.563 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-18T12:57:16.563 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:57:16.563 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-18T12:57:16.563 INFO:teuthology.orchestra.run.smithi167.stdout:monmaptool for a [v2:172.21.15.167:3300,v1:172.21.15.167:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-18T12:57:16.563 INFO:teuthology.orchestra.run.smithi167.stdout:setting min_mon_release = quincy 2024-06-18T12:57:16.564 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/monmaptool: set fsid to 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:57:16.564 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-18T12:57:16.564 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:57:16.564 INFO:teuthology.orchestra.run.smithi167.stdout:Creating mon... 2024-06-18T12:57:17.460 INFO:teuthology.orchestra.run.smithi167.stdout:create mon.a on 2024-06-18T12:57:17.653 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-18T12:57:17.794 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-18T12:57:17.964 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968.target → /etc/systemd/system/ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968.target. 2024-06-18T12:57:17.965 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968.target → /etc/systemd/system/ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968.target. 2024-06-18T12:57:18.216 INFO:teuthology.orchestra.run.smithi167.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.a 2024-06-18T12:57:18.216 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.a.service: Unit ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.a.service not loaded. 2024-06-18T12:57:18.380 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968.target.wants/ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.a.service → /etc/systemd/system/ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@.service. 2024-06-18T12:57:19.003 INFO:teuthology.orchestra.run.smithi167.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-18T12:57:19.004 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stdout disabled 2024-06-18T12:57:19.011 INFO:teuthology.orchestra.run.smithi167.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-18T12:57:19.011 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stdout inactive 2024-06-18T12:57:19.012 INFO:teuthology.orchestra.run.smithi167.stdout:firewalld.service is not enabled 2024-06-18T12:57:19.012 INFO:teuthology.orchestra.run.smithi167.stdout:Not possible to enable service . firewalld.service is not available 2024-06-18T12:57:19.012 INFO:teuthology.orchestra.run.smithi167.stdout:Waiting for mon to start... 2024-06-18T12:57:19.012 INFO:teuthology.orchestra.run.smithi167.stdout:Waiting for mon... 2024-06-18T12:57:19.137 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:18 smithi167 systemd[1]: Started Ceph mon.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T12:57:20.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout cluster: 2024-06-18T12:57:20.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout id: 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:57:20.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-18T12:57:20.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 2024-06-18T12:57:20.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout services: 2024-06-18T12:57:20.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.576586s) 2024-06-18T12:57:20.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-18T12:57:20.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-18T12:57:20.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 2024-06-18T12:57:20.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout data: 2024-06-18T12:57:20.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-18T12:57:20.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-18T12:57:20.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-18T12:57:20.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout pgs: 2024-06-18T12:57:20.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 2024-06-18T12:57:20.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.446+0000 7f460d9d1640 1 Processor -- start 2024-06-18T12:57:20.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.446+0000 7f460d9d1640 1 -- start start 2024-06-18T12:57:20.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.447+0000 7f460d9d1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4608104040 0x7f4608104440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:20.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.447+0000 7f460d9d1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4608104a10 con 0x7f4608104040 2024-06-18T12:57:20.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.447+0000 7f4606ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4608104040 0x7f4608104440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:20.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.447+0000 7f4606ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4608104040 0x7f4608104440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:57438/0 (socket says 172.21.15.167:57438) 2024-06-18T12:57:20.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.447+0000 7f4606ffd640 1 -- 172.21.15.167:0/1687155004 learned_addr learned my addr 172.21.15.167:0/1687155004 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:20.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.447+0000 7f4606ffd640 1 -- 172.21.15.167:0/1687155004 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4608105240 con 0x7f4608104040 2024-06-18T12:57:20.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.450+0000 7f4606ffd640 1 --2- 172.21.15.167:0/1687155004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4608104040 0x7f4608104440 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009820 tx=0x7f45f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e15ead0c98130b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:20.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.450+0000 7f4605ffb640 1 -- 172.21.15.167:0/1687155004 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45f8037070 con 0x7f4608104040 2024-06-18T12:57:20.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.450+0000 7f4605ffb640 1 -- 172.21.15.167:0/1687155004 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f8032440 con 0x7f4608104040 2024-06-18T12:57:20.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.450+0000 7f4605ffb640 1 -- 172.21.15.167:0/1687155004 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45f8032740 con 0x7f4608104040 2024-06-18T12:57:20.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.451+0000 7f460d9d1640 1 -- 172.21.15.167:0/1687155004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4608104040 msgr2=0x7f4608104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:20.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.451+0000 7f460d9d1640 1 --2- 172.21.15.167:0/1687155004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4608104040 0x7f4608104440 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009820 tx=0x7f45f802a270 comp rx=0 tx=0).stop 2024-06-18T12:57:20.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.451+0000 7f460d9d1640 1 -- 172.21.15.167:0/1687155004 shutdown_connections 2024-06-18T12:57:20.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.451+0000 7f460d9d1640 1 --2- 172.21.15.167:0/1687155004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4608104040 0x7f4608104440 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:20.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.451+0000 7f460d9d1640 1 -- 172.21.15.167:0/1687155004 >> 172.21.15.167:0/1687155004 conn(0x7f46080ff790 msgr2=0x7f4608101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:20.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.451+0000 7f460d9d1640 1 -- 172.21.15.167:0/1687155004 shutdown_connections 2024-06-18T12:57:20.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.451+0000 7f460d9d1640 1 -- 172.21.15.167:0/1687155004 wait complete. 2024-06-18T12:57:20.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.451+0000 7f460d9d1640 1 Processor -- start 2024-06-18T12:57:20.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.451+0000 7f460d9d1640 1 -- start start 2024-06-18T12:57:20.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.452+0000 7f460d9d1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4608104040 0x7f4608198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:20.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.452+0000 7f460d9d1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46081057e0 con 0x7f4608104040 2024-06-18T12:57:20.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.452+0000 7f4606ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4608104040 0x7f4608198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:20.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.452+0000 7f4606ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4608104040 0x7f4608198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:57446/0 (socket says 172.21.15.167:57446) 2024-06-18T12:57:20.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.452+0000 7f4606ffd640 1 -- 172.21.15.167:0/3987341640 learned_addr learned my addr 172.21.15.167:0/3987341640 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:20.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.452+0000 7f4606ffd640 1 -- 172.21.15.167:0/3987341640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4608198e40 con 0x7f4608104040 2024-06-18T12:57:20.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.452+0000 7f4606ffd640 1 --2- 172.21.15.167:0/3987341640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4608104040 0x7f4608198900 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f45f80090b0 tx=0x7f45f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:20.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.452+0000 7f460c9cf640 1 -- 172.21.15.167:0/3987341640 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45f8041070 con 0x7f4608104040 2024-06-18T12:57:20.049 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.452+0000 7f460c9cf640 1 -- 172.21.15.167:0/3987341640 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f803c7a0 con 0x7f4608104040 2024-06-18T12:57:20.049 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.452+0000 7f460d9d1640 1 -- 172.21.15.167:0/3987341640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46081990d0 con 0x7f4608104040 2024-06-18T12:57:20.049 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.452+0000 7f460c9cf640 1 -- 172.21.15.167:0/3987341640 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45f8037040 con 0x7f4608104040 2024-06-18T12:57:20.049 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.452+0000 7f460d9d1640 1 -- 172.21.15.167:0/3987341640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f460819bdc0 con 0x7f4608104040 2024-06-18T12:57:20.049 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.453+0000 7f460c9cf640 1 -- 172.21.15.167:0/3987341640 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f45f8044070 con 0x7f4608104040 2024-06-18T12:57:20.049 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.453+0000 7f460c9cf640 1 -- 172.21.15.167:0/3987341640 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f45f803d3f0 con 0x7f4608104040 2024-06-18T12:57:20.049 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.453+0000 7f460d9d1640 1 -- 172.21.15.167:0/3987341640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4608105e30 con 0x7f4608104040 2024-06-18T12:57:20.049 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.456+0000 7f460c9cf640 1 -- 172.21.15.167:0/3987341640 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f45f8046450 con 0x7f4608104040 2024-06-18T12:57:20.049 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.517+0000 7f460d9d1640 1 -- 172.21.15.167:0/3987341640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f4608191d40 con 0x7f4608104040 2024-06-18T12:57:20.049 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.518+0000 7f460c9cf640 1 -- 172.21.15.167:0/3987341640 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f45f8046630 con 0x7f4608104040 2024-06-18T12:57:20.050 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.518+0000 7f460d9d1640 1 -- 172.21.15.167:0/3987341640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4608104040 msgr2=0x7f4608198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:20.050 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.518+0000 7f460d9d1640 1 --2- 172.21.15.167:0/3987341640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4608104040 0x7f4608198900 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f45f80090b0 tx=0x7f45f80046c0 comp rx=0 tx=0).stop 2024-06-18T12:57:20.050 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.518+0000 7f460d9d1640 1 -- 172.21.15.167:0/3987341640 shutdown_connections 2024-06-18T12:57:20.050 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.518+0000 7f460d9d1640 1 --2- 172.21.15.167:0/3987341640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4608104040 0x7f4608198900 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:20.050 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.518+0000 7f460d9d1640 1 -- 172.21.15.167:0/3987341640 >> 172.21.15.167:0/3987341640 conn(0x7f46080ff790 msgr2=0x7f4608078850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:20.050 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.519+0000 7f460d9d1640 1 -- 172.21.15.167:0/3987341640 shutdown_connections 2024-06-18T12:57:20.050 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:19.519+0000 7f460d9d1640 1 -- 172.21.15.167:0/3987341640 wait complete. 2024-06-18T12:57:20.050 INFO:teuthology.orchestra.run.smithi167.stdout:mon is available 2024-06-18T12:57:20.050 INFO:teuthology.orchestra.run.smithi167.stdout:Assimilating anything we can from ceph.conf... 2024-06-18T12:57:20.153 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:19 smithi167 ceph-mon[29126]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-18T12:57:20.153 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:19 smithi167 ceph-mon[29126]: monmap epoch 1 2024-06-18T12:57:20.154 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:19 smithi167 ceph-mon[29126]: fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:57:20.154 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:19 smithi167 ceph-mon[29126]: last_changed 2024-06-18T12:57:16.004836+0000 2024-06-18T12:57:20.154 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:19 smithi167 ceph-mon[29126]: created 2024-06-18T12:57:16.004836+0000 2024-06-18T12:57:20.154 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:19 smithi167 ceph-mon[29126]: min_mon_release 19 (squid) 2024-06-18T12:57:20.154 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:19 smithi167 ceph-mon[29126]: election_strategy: 1 2024-06-18T12:57:20.154 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:19 smithi167 ceph-mon[29126]: 0: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.a 2024-06-18T12:57:20.154 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:19 smithi167 ceph-mon[29126]: fsmap 2024-06-18T12:57:20.154 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:19 smithi167 ceph-mon[29126]: osdmap e1: 0 total, 0 up, 0 in 2024-06-18T12:57:20.154 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:19 smithi167 ceph-mon[29126]: mgrmap e1: no daemons active 2024-06-18T12:57:20.154 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:19 smithi167 ceph-mon[29126]: from='client.? 172.21.15.167:0/3987341640' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-18T12:57:20.984 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 2024-06-18T12:57:20.984 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout [global] 2024-06-18T12:57:20.984 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout fsid = 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:57:20.984 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.167:3300,v1:172.21.15.167:6789] 2024-06-18T12:57:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-18T12:57:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-18T12:57:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-18T12:57:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-18T12:57:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 2024-06-18T12:57:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-18T12:57:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-18T12:57:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 2024-06-18T12:57:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout [osd] 2024-06-18T12:57:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-18T12:57:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-18T12:57:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.425+0000 7f3173fd3640 1 Processor -- start 2024-06-18T12:57:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.425+0000 7f3173fd3640 1 -- start start 2024-06-18T12:57:20.986 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.425+0000 7f3173fd3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c105110 0x7f316c105510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:20.986 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.425+0000 7f3173fd3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f316c105ae0 con 0x7f316c105110 2024-06-18T12:57:20.986 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.426+0000 7f3171d6f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c105110 0x7f316c105510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:20.986 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.426+0000 7f3171d6f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c105110 0x7f316c105510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:57454/0 (socket says 172.21.15.167:57454) 2024-06-18T12:57:20.986 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.426+0000 7f3171d6f640 1 -- 172.21.15.167:0/443931958 learned_addr learned my addr 172.21.15.167:0/443931958 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:20.986 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.426+0000 7f3171d6f640 1 -- 172.21.15.167:0/443931958 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316c1062c0 con 0x7f316c105110 2024-06-18T12:57:20.986 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.427+0000 7f3171d6f640 1 --2- 172.21.15.167:0/443931958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c105110 0x7f316c105510 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3168009820 tx=0x7f316802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f867688cb86830b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:20.986 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.427+0000 7f3170d6d640 1 -- 172.21.15.167:0/443931958 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3168037070 con 0x7f316c105110 2024-06-18T12:57:20.986 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.427+0000 7f3170d6d640 1 -- 172.21.15.167:0/443931958 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3168032440 con 0x7f316c105110 2024-06-18T12:57:20.986 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.427+0000 7f3170d6d640 1 -- 172.21.15.167:0/443931958 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3168032740 con 0x7f316c105110 2024-06-18T12:57:20.986 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.428+0000 7f3173fd3640 1 -- 172.21.15.167:0/443931958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c105110 msgr2=0x7f316c105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.428+0000 7f3173fd3640 1 --2- 172.21.15.167:0/443931958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c105110 0x7f316c105510 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3168009820 tx=0x7f316802a270 comp rx=0 tx=0).stop 2024-06-18T12:57:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.428+0000 7f3173fd3640 1 -- 172.21.15.167:0/443931958 shutdown_connections 2024-06-18T12:57:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.428+0000 7f3173fd3640 1 --2- 172.21.15.167:0/443931958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c105110 0x7f316c105510 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.428+0000 7f3173fd3640 1 -- 172.21.15.167:0/443931958 >> 172.21.15.167:0/443931958 conn(0x7f316c1008a0 msgr2=0x7f316c102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.428+0000 7f3173fd3640 1 -- 172.21.15.167:0/443931958 shutdown_connections 2024-06-18T12:57:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.428+0000 7f3173fd3640 1 -- 172.21.15.167:0/443931958 wait complete. 2024-06-18T12:57:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.428+0000 7f3173fd3640 1 Processor -- start 2024-06-18T12:57:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.428+0000 7f3173fd3640 1 -- start start 2024-06-18T12:57:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.429+0000 7f3173fd3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c105110 0x7f316c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.429+0000 7f3173fd3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f316c106860 con 0x7f316c105110 2024-06-18T12:57:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.429+0000 7f3171d6f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c105110 0x7f316c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.429+0000 7f3171d6f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c105110 0x7f316c198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:57464/0 (socket says 172.21.15.167:57464) 2024-06-18T12:57:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.429+0000 7f3171d6f640 1 -- 172.21.15.167:0/4146427457 learned_addr learned my addr 172.21.15.167:0/4146427457 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:20.988 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.429+0000 7f3171d6f640 1 -- 172.21.15.167:0/4146427457 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316c198da0 con 0x7f316c105110 2024-06-18T12:57:20.988 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.429+0000 7f3171d6f640 1 --2- 172.21.15.167:0/4146427457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c105110 0x7f316c198860 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f31680090b0 tx=0x7f31680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:20.988 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.429+0000 7f3162ffd640 1 -- 172.21.15.167:0/4146427457 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3168041070 con 0x7f316c105110 2024-06-18T12:57:20.988 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.429+0000 7f3162ffd640 1 -- 172.21.15.167:0/4146427457 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316803c7a0 con 0x7f316c105110 2024-06-18T12:57:20.988 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.429+0000 7f3173fd3640 1 -- 172.21.15.167:0/4146427457 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316c199030 con 0x7f316c105110 2024-06-18T12:57:20.988 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.429+0000 7f3162ffd640 1 -- 172.21.15.167:0/4146427457 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3168037040 con 0x7f316c105110 2024-06-18T12:57:20.988 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.429+0000 7f3173fd3640 1 -- 172.21.15.167:0/4146427457 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f316c19bd20 con 0x7f316c105110 2024-06-18T12:57:20.988 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.429+0000 7f3162ffd640 1 -- 172.21.15.167:0/4146427457 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3168044070 con 0x7f316c105110 2024-06-18T12:57:20.988 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.429+0000 7f3162ffd640 1 -- 172.21.15.167:0/4146427457 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f316803d3f0 con 0x7f316c105110 2024-06-18T12:57:20.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.430+0000 7f3173fd3640 1 -- 172.21.15.167:0/4146427457 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f316c106eb0 con 0x7f316c105110 2024-06-18T12:57:20.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.432+0000 7f3162ffd640 1 -- 172.21.15.167:0/4146427457 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3168046450 con 0x7f316c105110 2024-06-18T12:57:20.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.492+0000 7f3173fd3640 1 -- 172.21.15.167:0/4146427457 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f316c19d1f0 con 0x7f316c105110 2024-06-18T12:57:20.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.494+0000 7f3162ffd640 1 -- 172.21.15.167:0/4146427457 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+340 (secure 0 0 0) 0x7f3168046630 con 0x7f316c105110 2024-06-18T12:57:20.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.495+0000 7f3162ffd640 1 -- 172.21.15.167:0/4146427457 <== mon.0 v2:172.21.15.167:3300/0 8 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f3168056070 con 0x7f316c105110 2024-06-18T12:57:20.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.496+0000 7f3173fd3640 1 -- 172.21.15.167:0/4146427457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c105110 msgr2=0x7f316c198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:20.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.496+0000 7f3173fd3640 1 --2- 172.21.15.167:0/4146427457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c105110 0x7f316c198860 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f31680090b0 tx=0x7f31680046c0 comp rx=0 tx=0).stop 2024-06-18T12:57:20.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.496+0000 7f3173fd3640 1 -- 172.21.15.167:0/4146427457 shutdown_connections 2024-06-18T12:57:20.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.496+0000 7f3173fd3640 1 --2- 172.21.15.167:0/4146427457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c105110 0x7f316c198860 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:20.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.496+0000 7f3173fd3640 1 -- 172.21.15.167:0/4146427457 >> 172.21.15.167:0/4146427457 conn(0x7f316c1008a0 msgr2=0x7f316c18cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:20.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.496+0000 7f3173fd3640 1 -- 172.21.15.167:0/4146427457 shutdown_connections 2024-06-18T12:57:20.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:20.496+0000 7f3173fd3640 1 -- 172.21.15.167:0/4146427457 wait complete. 2024-06-18T12:57:20.989 INFO:teuthology.orchestra.run.smithi167.stdout:Generating new minimal ceph.conf... 2024-06-18T12:57:21.101 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:20 smithi167 ceph-mon[29126]: from='client.? 172.21.15.167:0/4146427457' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-18T12:57:21.101 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:20 smithi167 ceph-mon[29126]: from='client.? 172.21.15.167:0/4146427457' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-18T12:57:21.982 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:21 smithi167 ceph-mon[29126]: from='client.? 172.21.15.167:0/2754232907' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:57:21.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.364+0000 7f477c3ad640 1 Processor -- start 2024-06-18T12:57:21.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.364+0000 7f477c3ad640 1 -- start start 2024-06-18T12:57:21.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.364+0000 7f477c3ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4774105110 0x7f4774105510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:21.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.364+0000 7f477c3ad640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4774105ae0 con 0x7f4774105110 2024-06-18T12:57:21.989 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.364+0000 7f477a149640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4774105110 0x7f4774105510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:21.990 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.364+0000 7f477a149640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4774105110 0x7f4774105510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:57480/0 (socket says 172.21.15.167:57480) 2024-06-18T12:57:21.990 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.364+0000 7f477a149640 1 -- 172.21.15.167:0/2541042125 learned_addr learned my addr 172.21.15.167:0/2541042125 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:21.990 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.364+0000 7f477a149640 1 -- 172.21.15.167:0/2541042125 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47741062c0 con 0x7f4774105110 2024-06-18T12:57:21.990 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.365+0000 7f477a149640 1 --2- 172.21.15.167:0/2541042125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4774105110 0x7f4774105510 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4770009820 tx=0x7f477002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5febe4b7703b6c88 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:21.990 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.365+0000 7f4779147640 1 -- 172.21.15.167:0/2541042125 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4770037070 con 0x7f4774105110 2024-06-18T12:57:21.990 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.365+0000 7f4779147640 1 -- 172.21.15.167:0/2541042125 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f47700328a0 con 0x7f4774105110 2024-06-18T12:57:21.990 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.365+0000 7f4779147640 1 -- 172.21.15.167:0/2541042125 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4770030640 con 0x7f4774105110 2024-06-18T12:57:21.990 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.366+0000 7f477c3ad640 1 -- 172.21.15.167:0/2541042125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4774105110 msgr2=0x7f4774105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:21.990 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.366+0000 7f477c3ad640 1 --2- 172.21.15.167:0/2541042125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4774105110 0x7f4774105510 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4770009820 tx=0x7f477002a270 comp rx=0 tx=0).stop 2024-06-18T12:57:21.990 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.366+0000 7f477c3ad640 1 -- 172.21.15.167:0/2541042125 shutdown_connections 2024-06-18T12:57:21.990 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.366+0000 7f477c3ad640 1 --2- 172.21.15.167:0/2541042125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4774105110 0x7f4774105510 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:21.990 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.366+0000 7f477c3ad640 1 -- 172.21.15.167:0/2541042125 >> 172.21.15.167:0/2541042125 conn(0x7f47741008a0 msgr2=0x7f4774102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:21.991 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.366+0000 7f477c3ad640 1 -- 172.21.15.167:0/2541042125 shutdown_connections 2024-06-18T12:57:21.991 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.366+0000 7f477c3ad640 1 -- 172.21.15.167:0/2541042125 wait complete. 2024-06-18T12:57:21.991 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.366+0000 7f477c3ad640 1 Processor -- start 2024-06-18T12:57:21.991 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.367+0000 7f477c3ad640 1 -- start start 2024-06-18T12:57:21.991 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.367+0000 7f477c3ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4774105110 0x7f4774198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:21.991 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.367+0000 7f477c3ad640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f47741094f0 con 0x7f4774105110 2024-06-18T12:57:21.991 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.367+0000 7f477a149640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4774105110 0x7f4774198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:21.991 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.367+0000 7f477a149640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4774105110 0x7f4774198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:57492/0 (socket says 172.21.15.167:57492) 2024-06-18T12:57:21.991 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.367+0000 7f477a149640 1 -- 172.21.15.167:0/2754232907 learned_addr learned my addr 172.21.15.167:0/2754232907 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:21.991 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.367+0000 7f477a149640 1 -- 172.21.15.167:0/2754232907 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47741994e0 con 0x7f4774105110 2024-06-18T12:57:21.991 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.367+0000 7f477a149640 1 --2- 172.21.15.167:0/2754232907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4774105110 0x7f4774198fa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4770030380 tx=0x7f47700326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:21.991 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.367+0000 7f476b7fe640 1 -- 172.21.15.167:0/2754232907 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4770037040 con 0x7f4774105110 2024-06-18T12:57:21.992 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.367+0000 7f476b7fe640 1 -- 172.21.15.167:0/2754232907 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f4770040070 con 0x7f4774105110 2024-06-18T12:57:21.992 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.367+0000 7f477c3ad640 1 -- 172.21.15.167:0/2754232907 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4774199770 con 0x7f4774105110 2024-06-18T12:57:21.992 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.367+0000 7f476b7fe640 1 -- 172.21.15.167:0/2754232907 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f477003bc70 con 0x7f4774105110 2024-06-18T12:57:21.992 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.367+0000 7f477c3ad640 1 -- 172.21.15.167:0/2754232907 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f477419c460 con 0x7f4774105110 2024-06-18T12:57:21.992 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.368+0000 7f476b7fe640 1 -- 172.21.15.167:0/2754232907 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4770045460 con 0x7f4774105110 2024-06-18T12:57:21.992 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.368+0000 7f476b7fe640 1 -- 172.21.15.167:0/2754232907 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4770044690 con 0x7f4774105110 2024-06-18T12:57:21.992 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.368+0000 7f477c3ad640 1 -- 172.21.15.167:0/2754232907 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4774105590 con 0x7f4774105110 2024-06-18T12:57:21.992 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.370+0000 7f476b7fe640 1 -- 172.21.15.167:0/2754232907 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f477003b430 con 0x7f4774105110 2024-06-18T12:57:21.992 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.429+0000 7f477c3ad640 1 -- 172.21.15.167:0/2754232907 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f477419c800 con 0x7f4774105110 2024-06-18T12:57:21.993 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.430+0000 7f476b7fe640 1 -- 172.21.15.167:0/2754232907 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f477003b610 con 0x7f4774105110 2024-06-18T12:57:21.993 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.430+0000 7f477c3ad640 1 -- 172.21.15.167:0/2754232907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4774105110 msgr2=0x7f4774198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:21.993 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.430+0000 7f477c3ad640 1 --2- 172.21.15.167:0/2754232907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4774105110 0x7f4774198fa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4770030380 tx=0x7f47700326e0 comp rx=0 tx=0).stop 2024-06-18T12:57:21.993 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.431+0000 7f477c3ad640 1 -- 172.21.15.167:0/2754232907 shutdown_connections 2024-06-18T12:57:21.993 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.431+0000 7f477c3ad640 1 --2- 172.21.15.167:0/2754232907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4774105110 0x7f4774198fa0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:21.993 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.431+0000 7f477c3ad640 1 -- 172.21.15.167:0/2754232907 >> 172.21.15.167:0/2754232907 conn(0x7f47741008a0 msgr2=0x7f4774101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:21.993 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.431+0000 7f477c3ad640 1 -- 172.21.15.167:0/2754232907 shutdown_connections 2024-06-18T12:57:21.993 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:21.431+0000 7f477c3ad640 1 -- 172.21.15.167:0/2754232907 wait complete. 2024-06-18T12:57:21.993 INFO:teuthology.orchestra.run.smithi167.stdout:Restarting the monitor... 2024-06-18T12:57:22.253 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:22 smithi167 systemd[1]: Stopping Ceph mon.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T12:57:22.635 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:22 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a[29120]: 2024-06-18T12:57:22.253+0000 7f7a72391640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-18T12:57:22.635 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:22 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a[29120]: 2024-06-18T12:57:22.253+0000 7f7a72391640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-18T12:57:22.635 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:22 smithi167 podman[29369]: 2024-06-18 12:57:22.386562828 +0000 UTC m=+0.228973393 container died cacb0fbd417bf9dc0468e2b3388afead452722f59bca9b02dd6498841331ce60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, RELEASE=squid-3781e3a) 2024-06-18T12:57:22.886 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:22 smithi167 podman[29369]: 2024-06-18 12:57:22.71940319 +0000 UTC m=+0.561813752 container cleanup cacb0fbd417bf9dc0468e2b3388afead452722f59bca9b02dd6498841331ce60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a, io.buildah.version=1.36.0, RELEASE=squid-3781e3a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240612, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-18T12:57:22.886 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:22 smithi167 bash[29369]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a 2024-06-18T12:57:23.136 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:22 smithi167 podman[29382]: 2024-06-18 12:57:22.887364396 +0000 UTC m=+0.504789388 container remove cacb0fbd417bf9dc0468e2b3388afead452722f59bca9b02dd6498841331ce60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, RELEASE=squid-3781e3a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-18T12:57:23.427 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-18T12:57:23.428 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 systemd[1]: Stopped Ceph mon.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T12:57:23.428 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 systemd[1]: Starting Ceph mon.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T12:57:23.428 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 podman[29476]: 2024-06-18 12:57:23.321450676 +0000 UTC m=+0.083515982 container create 4e45951ebad496c4427e9aa68caa028740d64a8e343098ac6f001775737670d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, RELEASE=squid-3781e3a, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-18T12:57:23.428 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 podman[29476]: 2024-06-18 12:57:23.255234254 +0000 UTC m=+0.017299571 image pull af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 2024-06-18T12:57:23.602 INFO:teuthology.orchestra.run.smithi167.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-18T12:57:23.818 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 podman[29476]: 2024-06-18 12:57:23.523834598 +0000 UTC m=+0.285899917 container init 4e45951ebad496c4427e9aa68caa028740d64a8e343098ac6f001775737670d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a, io.buildah.version=1.36.0, org.label-schema.build-date=20240612, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, RELEASE=squid-3781e3a, ceph=True, org.label-schema.license=GPLv2) 2024-06-18T12:57:23.819 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 podman[29476]: 2024-06-18 12:57:23.527004249 +0000 UTC m=+0.289069546 container start 4e45951ebad496c4427e9aa68caa028740d64a8e343098ac6f001775737670d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-3781e3a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-18T12:57:23.819 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: set uid:gid to 167:167 (ceph:ceph) 2024-06-18T12:57:23.819 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: ceph version 19.0.0-3124-g3781e3a5 (3781e3a592387d16e8020f5db99cdee4198f6d00) squid (rc), process ceph-mon, pid 2 2024-06-18T12:57:23.819 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: pidfile_write: ignore empty --pid-file 2024-06-18T12:57:23.819 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: load: jerasure load: lrc 2024-06-18T12:57:23.819 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: RocksDB version: 7.9.2 2024-06-18T12:57:23.819 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Git sha 0 2024-06-18T12:57:23.819 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Compile date 2024-06-17 15:21:02 2024-06-18T12:57:23.819 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: DB SUMMARY 2024-06-18T12:57:23.819 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: DB Session ID: 41DGSMIIVXPJGGYF2E73 2024-06-18T12:57:23.819 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: CURRENT file: CURRENT 2024-06-18T12:57:23.819 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: IDENTITY file: IDENTITY 2024-06-18T12:57:23.820 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-18T12:57:23.820 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-18T12:57:23.820 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 90987 ; 2024-06-18T12:57:23.820 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.error_if_exists: 0 2024-06-18T12:57:23.820 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.create_if_missing: 0 2024-06-18T12:57:23.820 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.paranoid_checks: 1 2024-06-18T12:57:23.820 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-18T12:57:23.820 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-18T12:57:23.820 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-18T12:57:23.820 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.env: 0x55abddf0a740 2024-06-18T12:57:23.821 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.fs: PosixFileSystem 2024-06-18T12:57:23.821 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.info_log: 0x55abde94f000 2024-06-18T12:57:23.821 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_file_opening_threads: 16 2024-06-18T12:57:23.821 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.statistics: (nil) 2024-06-18T12:57:23.821 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.use_fsync: 0 2024-06-18T12:57:23.821 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_log_file_size: 0 2024-06-18T12:57:23.821 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-18T12:57:23.821 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-18T12:57:23.821 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.keep_log_file_num: 1000 2024-06-18T12:57:23.821 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.recycle_log_file_num: 0 2024-06-18T12:57:23.821 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.allow_fallocate: 1 2024-06-18T12:57:23.821 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.allow_mmap_reads: 0 2024-06-18T12:57:23.821 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.allow_mmap_writes: 0 2024-06-18T12:57:23.821 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.use_direct_reads: 0 2024-06-18T12:57:23.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-18T12:57:23.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.create_missing_column_families: 0 2024-06-18T12:57:23.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.db_log_dir: 2024-06-18T12:57:23.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.wal_dir: 2024-06-18T12:57:23.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-18T12:57:23.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-18T12:57:23.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-18T12:57:23.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-18T12:57:23.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-18T12:57:23.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-18T12:57:23.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.advise_random_on_open: 1 2024-06-18T12:57:23.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.db_write_buffer_size: 0 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.write_buffer_manager: 0x55abde95e1e0 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.rate_limiter: (nil) 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.wal_recovery_mode: 2 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.enable_thread_tracking: 0 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.enable_pipelined_write: 0 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.unordered_write: 0 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-18T12:57:23.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.row_cache: None 2024-06-18T12:57:23.824 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.wal_filter: None 2024-06-18T12:57:23.824 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-18T12:57:23.825 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.allow_ingest_behind: 0 2024-06-18T12:57:23.825 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.two_write_queues: 0 2024-06-18T12:57:23.825 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.manual_wal_flush: 0 2024-06-18T12:57:23.825 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.wal_compression: 0 2024-06-18T12:57:23.825 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.atomic_flush: 0 2024-06-18T12:57:23.825 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-18T12:57:23.825 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-18T12:57:23.825 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-18T12:57:23.825 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.log_readahead_size: 0 2024-06-18T12:57:23.825 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-18T12:57:23.826 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.best_efforts_recovery: 0 2024-06-18T12:57:23.826 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-18T12:57:23.826 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-18T12:57:23.826 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.allow_data_in_errors: 0 2024-06-18T12:57:23.826 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.db_host_id: __hostname__ 2024-06-18T12:57:23.826 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-18T12:57:23.826 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_background_jobs: 2 2024-06-18T12:57:23.826 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_background_compactions: -1 2024-06-18T12:57:23.826 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_subcompactions: 1 2024-06-18T12:57:23.826 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-18T12:57:23.826 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-18T12:57:23.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-18T12:57:23.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_total_wal_size: 0 2024-06-18T12:57:23.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-18T12:57:23.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-18T12:57:23.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-18T12:57:23.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-18T12:57:23.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_open_files: -1 2024-06-18T12:57:23.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.bytes_per_sync: 0 2024-06-18T12:57:23.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-18T12:57:23.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-18T12:57:23.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compaction_readahead_size: 0 2024-06-18T12:57:23.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_background_flushes: -1 2024-06-18T12:57:23.828 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Compression algorithms supported: 2024-06-18T12:57:23.828 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: kZSTD supported: 0 2024-06-18T12:57:23.828 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: kXpressCompression supported: 0 2024-06-18T12:57:23.828 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: kBZip2Compression supported: 0 2024-06-18T12:57:23.828 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-18T12:57:23.828 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: kLZ4Compression supported: 1 2024-06-18T12:57:23.828 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: kZlibCompression supported: 1 2024-06-18T12:57:23.828 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: kLZ4HCCompression supported: 1 2024-06-18T12:57:23.828 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: kSnappyCompression supported: 1 2024-06-18T12:57:23.828 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-18T12:57:23.828 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-18T12:57:23.828 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-18T12:57:23.828 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-18T12:57:23.828 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.merge_operator: 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compaction_filter: None 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compaction_filter_factory: None 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.sst_partitioner_factory: None 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55abde94ed00) 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout: cache_index_and_filter_blocks: 1 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout: pin_top_level_index_and_filter: 1 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout: index_type: 0 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout: data_block_index_type: 0 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout: index_shortening: 1 2024-06-18T12:57:23.829 INFO:journalctl@ceph.mon.a.smithi167.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-18T12:57:23.830 INFO:journalctl@ceph.mon.a.smithi167.stdout: checksum: 4 2024-06-18T12:57:23.830 INFO:journalctl@ceph.mon.a.smithi167.stdout: no_block_cache: 0 2024-06-18T12:57:23.830 INFO:journalctl@ceph.mon.a.smithi167.stdout: block_cache: 0x55abde945090 2024-06-18T12:57:23.830 INFO:journalctl@ceph.mon.a.smithi167.stdout: block_cache_name: BinnedLRUCache 2024-06-18T12:57:23.830 INFO:journalctl@ceph.mon.a.smithi167.stdout: block_cache_options: 2024-06-18T12:57:23.830 INFO:journalctl@ceph.mon.a.smithi167.stdout: capacity : 536870912 2024-06-18T12:57:23.830 INFO:journalctl@ceph.mon.a.smithi167.stdout: num_shard_bits : 4 2024-06-18T12:57:23.830 INFO:journalctl@ceph.mon.a.smithi167.stdout: strict_capacity_limit : 0 2024-06-18T12:57:23.830 INFO:journalctl@ceph.mon.a.smithi167.stdout: high_pri_pool_ratio: 0.000 2024-06-18T12:57:23.830 INFO:journalctl@ceph.mon.a.smithi167.stdout: block_cache_compressed: (nil) 2024-06-18T12:57:23.830 INFO:journalctl@ceph.mon.a.smithi167.stdout: persistent_cache: (nil) 2024-06-18T12:57:23.830 INFO:journalctl@ceph.mon.a.smithi167.stdout: block_size: 4096 2024-06-18T12:57:23.830 INFO:journalctl@ceph.mon.a.smithi167.stdout: block_size_deviation: 10 2024-06-18T12:57:23.830 INFO:journalctl@ceph.mon.a.smithi167.stdout: block_restart_interval: 16 2024-06-18T12:57:23.831 INFO:journalctl@ceph.mon.a.smithi167.stdout: index_block_restart_interval: 1 2024-06-18T12:57:23.831 INFO:journalctl@ceph.mon.a.smithi167.stdout: metadata_block_size: 4096 2024-06-18T12:57:23.831 INFO:journalctl@ceph.mon.a.smithi167.stdout: partition_filters: 0 2024-06-18T12:57:23.831 INFO:journalctl@ceph.mon.a.smithi167.stdout: use_delta_encoding: 1 2024-06-18T12:57:23.831 INFO:journalctl@ceph.mon.a.smithi167.stdout: filter_policy: bloomfilter 2024-06-18T12:57:23.831 INFO:journalctl@ceph.mon.a.smithi167.stdout: whole_key_filtering: 1 2024-06-18T12:57:23.831 INFO:journalctl@ceph.mon.a.smithi167.stdout: verify_compression: 0 2024-06-18T12:57:23.832 INFO:journalctl@ceph.mon.a.smithi167.stdout: read_amp_bytes_per_bit: 0 2024-06-18T12:57:23.832 INFO:journalctl@ceph.mon.a.smithi167.stdout: format_version: 5 2024-06-18T12:57:23.832 INFO:journalctl@ceph.mon.a.smithi167.stdout: enable_index_compression: 1 2024-06-18T12:57:23.832 INFO:journalctl@ceph.mon.a.smithi167.stdout: block_align: 0 2024-06-18T12:57:23.832 INFO:journalctl@ceph.mon.a.smithi167.stdout: max_auto_readahead_size: 262144 2024-06-18T12:57:23.832 INFO:journalctl@ceph.mon.a.smithi167.stdout: prepopulate_block_cache: 0 2024-06-18T12:57:23.832 INFO:journalctl@ceph.mon.a.smithi167.stdout: initial_auto_readahead_size: 8192 2024-06-18T12:57:23.832 INFO:journalctl@ceph.mon.a.smithi167.stdout: num_file_reads_for_auto_readahead: 2 2024-06-18T12:57:23.832 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.write_buffer_size: 33554432 2024-06-18T12:57:23.832 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_write_buffer_number: 2 2024-06-18T12:57:23.832 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compression: NoCompression 2024-06-18T12:57:23.832 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.bottommost_compression: Disabled 2024-06-18T12:57:23.832 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.prefix_extractor: nullptr 2024-06-18T12:57:23.833 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-18T12:57:23.833 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.num_levels: 7 2024-06-18T12:57:23.833 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-18T12:57:23.833 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-18T12:57:23.833 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-18T12:57:23.833 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-18T12:57:23.833 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-18T12:57:23.833 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-18T12:57:23.833 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-18T12:57:23.833 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-18T12:57:23.833 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-18T12:57:23.833 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-18T12:57:23.833 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-18T12:57:23.833 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compression_opts.level: 32767 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compression_opts.strategy: 0 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compression_opts.enabled: false 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.target_file_size_base: 67108864 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-18T12:57:23.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.arena_block_size: 1048576 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-18T12:57:23.835 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-18T12:57:23.836 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.disable_auto_compactions: 0 2024-06-18T12:57:23.836 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-18T12:57:23.836 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-18T12:57:23.836 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-18T12:57:23.836 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-18T12:57:23.836 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-18T12:57:23.836 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-18T12:57:23.836 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-18T12:57:23.836 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-18T12:57:23.836 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-18T12:57:23.836 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-18T12:57:23.836 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.table_properties_collectors: 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.inplace_update_support: 0 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.bloom_locality: 0 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.max_successive_merges: 0 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.paranoid_file_checks: 0 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.force_consistency_checks: 1 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.report_bg_io_stats: 0 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.ttl: 2592000 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-18T12:57:23.837 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-18T12:57:23.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.enable_blob_files: false 2024-06-18T12:57:23.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.min_blob_size: 0 2024-06-18T12:57:23.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.blob_file_size: 268435456 2024-06-18T12:57:23.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-18T12:57:23.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-18T12:57:23.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-18T12:57:23.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-18T12:57:23.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-18T12:57:23.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.blob_file_starting_level: 0 2024-06-18T12:57:23.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-18T12:57:23.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-18T12:57:23.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-18T12:57:23.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6a9892c5-5f3e-44d2-aee8-100528237e9b 2024-06-18T12:57:23.839 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718715443553132, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-18T12:57:23.839 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-18T12:57:23.839 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718715443553992, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 87375, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 268, "table_properties": {"data_size": 85533, "index_size": 252, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10766, "raw_average_key_size": 47, "raw_value_size": 79443, "raw_average_value_size": 349, "num_data_blocks": 11, "num_entries": 227, "num_filter_entries": 227, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1718715443, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6a9892c5-5f3e-44d2-aee8-100528237e9b", "db_session_id": "41DGSMIIVXPJGGYF2E73", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-18T12:57:23.839 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718715443554089, "job": 1, "event": "recovery_finished"} 2024-06-18T12:57:23.839 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-18T12:57:23.839 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-18T12:57:23.839 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55abde972e00 2024-06-18T12:57:23.839 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: DB pointer 0x55abdea68000 2024-06-18T12:57:23.839 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-18T12:57:23.839 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-18T12:57:23.839 INFO:journalctl@ceph.mon.a.smithi167.stdout: ** DB Stats ** 2024-06-18T12:57:23.839 INFO:journalctl@ceph.mon.a.smithi167.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-18T12:57:23.839 INFO:journalctl@ceph.mon.a.smithi167.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: ** Compaction Stats [default] ** 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: L0 2/0 87.16 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: Sum 2/0 87.16 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: ** Compaction Stats [default] ** 2024-06-18T12:57:23.840 INFO:journalctl@ceph.mon.a.smithi167.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-18T12:57:23.841 INFO:journalctl@ceph.mon.a.smithi167.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-18T12:57:23.841 INFO:journalctl@ceph.mon.a.smithi167.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 151.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-18T12:57:23.841 INFO:journalctl@ceph.mon.a.smithi167.stdout: 2024-06-18T12:57:23.841 INFO:journalctl@ceph.mon.a.smithi167.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-18T12:57:23.841 INFO:journalctl@ceph.mon.a.smithi167.stdout: 2024-06-18T12:57:23.841 INFO:journalctl@ceph.mon.a.smithi167.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-18T12:57:23.841 INFO:journalctl@ceph.mon.a.smithi167.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-18T12:57:23.841 INFO:journalctl@ceph.mon.a.smithi167.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-18T12:57:23.841 INFO:journalctl@ceph.mon.a.smithi167.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-18T12:57:23.841 INFO:journalctl@ceph.mon.a.smithi167.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-18T12:57:23.841 INFO:journalctl@ceph.mon.a.smithi167.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-18T12:57:23.841 INFO:journalctl@ceph.mon.a.smithi167.stdout: Cumulative compaction: 0.00 GB write, 28.38 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-18T12:57:23.841 INFO:journalctl@ceph.mon.a.smithi167.stdout: Interval compaction: 0.00 GB write, 28.38 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-18T12:57:23.841 INFO:journalctl@ceph.mon.a.smithi167.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-18T12:57:23.842 INFO:journalctl@ceph.mon.a.smithi167.stdout: Block cache BinnedLRUCache@0x55abde945090#2 capacity: 512.00 MB usage: 1.20 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-18T12:57:23.842 INFO:journalctl@ceph.mon.a.smithi167.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-06-18T12:57:23.842 INFO:journalctl@ceph.mon.a.smithi167.stdout: 2024-06-18T12:57:23.842 INFO:journalctl@ceph.mon.a.smithi167.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-18T12:57:23.842 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: starting mon.a rank 0 at public addrs [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] at bind addrs [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:57:23.842 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: mon.a@-1(???) e1 preinit fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:57:23.842 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: mon.a@-1(???).mds e1 new map 2024-06-18T12:57:23.842 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: mon.a@-1(???).mds e1 print_map 2024-06-18T12:57:23.842 INFO:journalctl@ceph.mon.a.smithi167.stdout: e1 2024-06-18T12:57:23.842 INFO:journalctl@ceph.mon.a.smithi167.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-18T12:57:23.842 INFO:journalctl@ceph.mon.a.smithi167.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-18T12:57:23.842 INFO:journalctl@ceph.mon.a.smithi167.stdout: legacy client fscid: -1 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout: 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout: No filesystems configured 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 bash[29476]: 4e45951ebad496c4427e9aa68caa028740d64a8e343098ac6f001775737670d5 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 systemd[1]: Started Ceph mon.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: monmap epoch 1 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: last_changed 2024-06-18T12:57:16.004836+0000 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: created 2024-06-18T12:57:16.004836+0000 2024-06-18T12:57:23.843 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: min_mon_release 19 (squid) 2024-06-18T12:57:23.844 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: election_strategy: 1 2024-06-18T12:57:23.844 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: 0: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.a 2024-06-18T12:57:23.844 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: fsmap 2024-06-18T12:57:23.844 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: osdmap e1: 0 total, 0 up, 0 in 2024-06-18T12:57:23.844 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:23 smithi167 ceph-mon[29496]: mgrmap e1: no daemons active 2024-06-18T12:57:24.611 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.015+0000 7fc47cfa2640 1 Processor -- start 2024-06-18T12:57:24.611 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.015+0000 7fc47cfa2640 1 -- start start 2024-06-18T12:57:24.611 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.016+0000 7fc47cfa2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478103e00 0x7fc478104200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:24.611 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.016+0000 7fc47cfa2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc478104740 con 0x7fc478103e00 2024-06-18T12:57:24.611 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.016+0000 7fc4777fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478103e00 0x7fc478104200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:24.611 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.016+0000 7fc4777fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478103e00 0x7fc478104200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45994/0 (socket says 172.21.15.167:45994) 2024-06-18T12:57:24.611 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.016+0000 7fc4777fe640 1 -- 172.21.15.167:0/1518699110 learned_addr learned my addr 172.21.15.167:0/1518699110 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:24.611 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.016+0000 7fc4777fe640 1 -- 172.21.15.167:0/1518699110 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc478104f60 con 0x7fc478103e00 2024-06-18T12:57:24.611 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.018+0000 7fc4777fe640 1 --2- 172.21.15.167:0/1518699110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478103e00 0x7fc478104200 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc468009820 tx=0x7fc46802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514fbe034f4a1184 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:24.611 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.018+0000 7fc4767fc640 1 -- 172.21.15.167:0/1518699110 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc468037070 con 0x7fc478103e00 2024-06-18T12:57:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.018+0000 7fc4767fc640 1 -- 172.21.15.167:0/1518699110 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7fc4680328a0 con 0x7fc478103e00 2024-06-18T12:57:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.018+0000 7fc4767fc640 1 -- 172.21.15.167:0/1518699110 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc468030640 con 0x7fc478103e00 2024-06-18T12:57:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.019+0000 7fc47cfa2640 1 -- 172.21.15.167:0/1518699110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478103e00 msgr2=0x7fc478104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.019+0000 7fc47cfa2640 1 --2- 172.21.15.167:0/1518699110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478103e00 0x7fc478104200 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc468009820 tx=0x7fc46802a270 comp rx=0 tx=0).stop 2024-06-18T12:57:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.019+0000 7fc47cfa2640 1 -- 172.21.15.167:0/1518699110 shutdown_connections 2024-06-18T12:57:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.019+0000 7fc47cfa2640 1 --2- 172.21.15.167:0/1518699110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478103e00 0x7fc478104200 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.019+0000 7fc47cfa2640 1 -- 172.21.15.167:0/1518699110 >> 172.21.15.167:0/1518699110 conn(0x7fc4780ff590 msgr2=0x7fc4781019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.019+0000 7fc47cfa2640 1 -- 172.21.15.167:0/1518699110 shutdown_connections 2024-06-18T12:57:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.019+0000 7fc47cfa2640 1 -- 172.21.15.167:0/1518699110 wait complete. 2024-06-18T12:57:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.019+0000 7fc47cfa2640 1 Processor -- start 2024-06-18T12:57:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.020+0000 7fc47cfa2640 1 -- start start 2024-06-18T12:57:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.020+0000 7fc47cfa2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478103e00 0x7fc4780789d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.020+0000 7fc47cfa2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc478108220 con 0x7fc478103e00 2024-06-18T12:57:24.613 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.020+0000 7fc4777fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478103e00 0x7fc4780789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:24.613 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.020+0000 7fc4777fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478103e00 0x7fc4780789d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46004/0 (socket says 172.21.15.167:46004) 2024-06-18T12:57:24.613 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.020+0000 7fc4777fe640 1 -- 172.21.15.167:0/85089654 learned_addr learned my addr 172.21.15.167:0/85089654 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:24.613 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.020+0000 7fc4777fe640 1 -- 172.21.15.167:0/85089654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47819baf0 con 0x7fc478103e00 2024-06-18T12:57:24.613 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.020+0000 7fc4777fe640 1 --2- 172.21.15.167:0/85089654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478103e00 0x7fc4780789d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc468030380 tx=0x7fc4680326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:24.613 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.020+0000 7fc474ff9640 1 -- 172.21.15.167:0/85089654 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc468037040 con 0x7fc478103e00 2024-06-18T12:57:24.613 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.020+0000 7fc47cfa2640 1 -- 172.21.15.167:0/85089654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47819bcc0 con 0x7fc478103e00 2024-06-18T12:57:24.613 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.020+0000 7fc474ff9640 1 -- 172.21.15.167:0/85089654 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7fc468040070 con 0x7fc478103e00 2024-06-18T12:57:24.613 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.020+0000 7fc474ff9640 1 -- 172.21.15.167:0/85089654 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc46803bca0 con 0x7fc478103e00 2024-06-18T12:57:24.613 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.020+0000 7fc47cfa2640 1 -- 172.21.15.167:0/85089654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47819c060 con 0x7fc478103e00 2024-06-18T12:57:24.613 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.021+0000 7fc474ff9640 1 -- 172.21.15.167:0/85089654 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc468045460 con 0x7fc478103e00 2024-06-18T12:57:24.613 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.021+0000 7fc474ff9640 1 -- 172.21.15.167:0/85089654 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc468044720 con 0x7fc478103e00 2024-06-18T12:57:24.613 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.021+0000 7fc47cfa2640 1 -- 172.21.15.167:0/85089654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4781083a0 con 0x7fc478103e00 2024-06-18T12:57:24.613 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.024+0000 7fc474ff9640 1 -- 172.21.15.167:0/85089654 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc46803b430 con 0x7fc478103e00 2024-06-18T12:57:24.614 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.083+0000 7fc47cfa2640 1 -- 172.21.15.167:0/85089654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fc47819c2e0 con 0x7fc478103e00 2024-06-18T12:57:24.614 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.084+0000 7fc474ff9640 1 -- 172.21.15.167:0/85089654 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fc46803b610 con 0x7fc478103e00 2024-06-18T12:57:24.614 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.085+0000 7fc47cfa2640 1 -- 172.21.15.167:0/85089654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478103e00 msgr2=0x7fc4780789d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:24.614 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.085+0000 7fc47cfa2640 1 --2- 172.21.15.167:0/85089654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478103e00 0x7fc4780789d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc468030380 tx=0x7fc4680326e0 comp rx=0 tx=0).stop 2024-06-18T12:57:24.614 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.085+0000 7fc47cfa2640 1 -- 172.21.15.167:0/85089654 shutdown_connections 2024-06-18T12:57:24.614 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.085+0000 7fc47cfa2640 1 --2- 172.21.15.167:0/85089654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478103e00 0x7fc4780789d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:24.614 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.085+0000 7fc47cfa2640 1 -- 172.21.15.167:0/85089654 >> 172.21.15.167:0/85089654 conn(0x7fc4780ff590 msgr2=0x7fc478100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:24.614 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.086+0000 7fc47cfa2640 1 -- 172.21.15.167:0/85089654 shutdown_connections 2024-06-18T12:57:24.614 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:24.086+0000 7fc47cfa2640 1 -- 172.21.15.167:0/85089654 wait complete. 2024-06-18T12:57:24.614 INFO:teuthology.orchestra.run.smithi167.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-18T12:57:24.628 INFO:teuthology.orchestra.run.smithi167.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-18T12:57:24.629 INFO:teuthology.orchestra.run.smithi167.stdout:Creating mgr... 2024-06-18T12:57:24.629 INFO:teuthology.orchestra.run.smithi167.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-18T12:57:24.629 INFO:teuthology.orchestra.run.smithi167.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-18T12:57:24.878 INFO:teuthology.orchestra.run.smithi167.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mgr.y 2024-06-18T12:57:24.878 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mgr.y.service: Unit ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mgr.y.service not loaded. 2024-06-18T12:57:25.029 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968.target.wants/ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@.service. 2024-06-18T12:57:25.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/85089654' entity='client.admin' 2024-06-18T12:57:25.567 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:25 smithi167 podman[29709]: 2024-06-18 12:57:25.470673 +0000 UTC m=+0.316005225 container start c2b474c3156f158284c9371bd88226d874c28ac90d9e5a95b8dd8ca08fdd575d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3781e3a, ceph=True) 2024-06-18T12:57:25.567 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:25 smithi167 bash[29709]: c2b474c3156f158284c9371bd88226d874c28ac90d9e5a95b8dd8ca08fdd575d 2024-06-18T12:57:25.581 INFO:teuthology.orchestra.run.smithi167.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-18T12:57:25.581 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stdout disabled 2024-06-18T12:57:25.588 INFO:teuthology.orchestra.run.smithi167.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-18T12:57:25.589 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stdout inactive 2024-06-18T12:57:25.589 INFO:teuthology.orchestra.run.smithi167.stdout:firewalld.service is not enabled 2024-06-18T12:57:25.589 INFO:teuthology.orchestra.run.smithi167.stdout:Not possible to enable service . firewalld.service is not available 2024-06-18T12:57:25.598 INFO:teuthology.orchestra.run.smithi167.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-18T12:57:25.598 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stdout disabled 2024-06-18T12:57:25.606 INFO:teuthology.orchestra.run.smithi167.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-18T12:57:25.606 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stdout inactive 2024-06-18T12:57:25.607 INFO:teuthology.orchestra.run.smithi167.stdout:firewalld.service is not enabled 2024-06-18T12:57:25.607 INFO:teuthology.orchestra.run.smithi167.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-18T12:57:25.607 INFO:teuthology.orchestra.run.smithi167.stdout:Waiting for mgr to start... 2024-06-18T12:57:25.607 INFO:teuthology.orchestra.run.smithi167.stdout:Waiting for mgr... 2024-06-18T12:57:25.830 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:25 smithi167 systemd[1]: Started Ceph mgr.y for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T12:57:25.831 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:25.599+0000 7f2c4d96d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-18T12:57:25.831 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:25.760+0000 7f2c4d96d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-18T12:57:25.831 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:25.831+0000 7f2c4d96d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-18T12:57:26.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2176259123' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-18T12:57:26.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:26 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:26.095+0000 7f2c4d96d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-18T12:57:26.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:26 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:26.160+0000 7f2c4d96d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-18T12:57:26.664 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 2024-06-18T12:57:26.664 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout { 2024-06-18T12:57:26.664 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "fsid": "3434f1e2-2d72-11ef-bc9d-c7b262605968", 2024-06-18T12:57:26.664 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "health": { 2024-06-18T12:57:26.664 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-18T12:57:26.664 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-18T12:57:26.664 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-18T12:57:26.665 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:26.665 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-18T12:57:26.665 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-18T12:57:26.665 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 0 2024-06-18T12:57:26.665 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout ], 2024-06-18T12:57:26.665 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-18T12:57:26.665 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "a" 2024-06-18T12:57:26.665 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout ], 2024-06-18T12:57:26.665 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-06-18T12:57:26.665 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-18T12:57:26.665 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-18T12:57:26.665 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-18T12:57:26.666 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-18T12:57:26.666 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:26.666 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-18T12:57:26.666 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-18T12:57:26.666 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-18T12:57:26.666 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-18T12:57:26.666 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-18T12:57:26.666 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-18T12:57:26.666 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-18T12:57:26.666 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-18T12:57:26.667 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:26.667 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-18T12:57:26.667 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-18T12:57:26.667 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-18T12:57:26.667 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-18T12:57:26.667 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-18T12:57:26.667 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-18T12:57:26.667 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-18T12:57:26.667 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-18T12:57:26.667 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-18T12:57:26.667 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:26.667 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-18T12:57:26.668 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-18T12:57:26.668 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-18T12:57:26.668 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-18T12:57:26.668 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:26.668 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-18T12:57:26.668 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-18T12:57:26.668 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-18T12:57:26.668 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-18T12:57:26.668 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-18T12:57:26.668 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-18T12:57:26.669 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "restful" 2024-06-18T12:57:26.669 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout ], 2024-06-18T12:57:26.669 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-18T12:57:26.669 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:26.669 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-18T12:57:26.669 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-18T12:57:26.669 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "modified": "2024-06-18T12:57:18.946357+0000", 2024-06-18T12:57:26.669 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-18T12:57:26.669 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:26.669 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-18T12:57:26.669 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout } 2024-06-18T12:57:26.669 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.009+0000 7fdaa8d08640 1 Processor -- start 2024-06-18T12:57:26.669 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.009+0000 7fdaa8d08640 1 -- start start 2024-06-18T12:57:26.669 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.010+0000 7fdaa8d08640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4105bb0 0x7fdaa4107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:26.670 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.010+0000 7fdaa8d08640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa4078e30 con 0x7fdaa4105bb0 2024-06-18T12:57:26.670 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.010+0000 7fdaa259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4105bb0 0x7fdaa4107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:26.670 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.010+0000 7fdaa259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4105bb0 0x7fdaa4107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46022/0 (socket says 172.21.15.167:46022) 2024-06-18T12:57:26.670 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.010+0000 7fdaa259c640 1 -- 172.21.15.167:0/4153262644 learned_addr learned my addr 172.21.15.167:0/4153262644 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:26.670 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.010+0000 7fdaa259c640 1 -- 172.21.15.167:0/4153262644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa41084e0 con 0x7fdaa4105bb0 2024-06-18T12:57:26.670 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.011+0000 7fdaa259c640 1 --2- 172.21.15.167:0/4153262644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4105bb0 0x7fdaa4107fa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fda94009820 tx=0x7fda9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe936d5d07b274f server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:26.670 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.011+0000 7fdaa159a640 1 -- 172.21.15.167:0/4153262644 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda94037070 con 0x7fdaa4105bb0 2024-06-18T12:57:26.670 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.011+0000 7fdaa159a640 1 -- 172.21.15.167:0/4153262644 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7fda940328a0 con 0x7fdaa4105bb0 2024-06-18T12:57:26.670 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.011+0000 7fdaa159a640 1 -- 172.21.15.167:0/4153262644 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda94030640 con 0x7fdaa4105bb0 2024-06-18T12:57:26.670 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.012+0000 7fdaa8d08640 1 -- 172.21.15.167:0/4153262644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4105bb0 msgr2=0x7fdaa4107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:26.671 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.012+0000 7fdaa8d08640 1 --2- 172.21.15.167:0/4153262644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4105bb0 0x7fdaa4107fa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fda94009820 tx=0x7fda9402a270 comp rx=0 tx=0).stop 2024-06-18T12:57:26.671 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.012+0000 7fdaa8d08640 1 -- 172.21.15.167:0/4153262644 shutdown_connections 2024-06-18T12:57:26.671 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.012+0000 7fdaa8d08640 1 --2- 172.21.15.167:0/4153262644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4105bb0 0x7fdaa4107fa0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:26.671 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.012+0000 7fdaa8d08640 1 -- 172.21.15.167:0/4153262644 >> 172.21.15.167:0/4153262644 conn(0x7fdaa40ff790 msgr2=0x7fdaa4101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:26.671 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.012+0000 7fdaa8d08640 1 -- 172.21.15.167:0/4153262644 shutdown_connections 2024-06-18T12:57:26.671 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.012+0000 7fdaa8d08640 1 -- 172.21.15.167:0/4153262644 wait complete. 2024-06-18T12:57:26.671 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.013+0000 7fdaa8d08640 1 Processor -- start 2024-06-18T12:57:26.671 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.013+0000 7fdaa8d08640 1 -- start start 2024-06-18T12:57:26.671 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.013+0000 7fdaa8d08640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4105bb0 0x7fdaa4199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:26.671 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.013+0000 7fdaa8d08640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa4109460 con 0x7fdaa4105bb0 2024-06-18T12:57:26.672 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.013+0000 7fdaa259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4105bb0 0x7fdaa4199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:26.672 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.013+0000 7fdaa259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4105bb0 0x7fdaa4199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46032/0 (socket says 172.21.15.167:46032) 2024-06-18T12:57:26.672 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.013+0000 7fdaa259c640 1 -- 172.21.15.167:0/2176259123 learned_addr learned my addr 172.21.15.167:0/2176259123 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:26.672 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.014+0000 7fdaa259c640 1 -- 172.21.15.167:0/2176259123 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa41995c0 con 0x7fdaa4105bb0 2024-06-18T12:57:26.672 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.014+0000 7fdaa259c640 1 --2- 172.21.15.167:0/2176259123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4105bb0 0x7fdaa4199080 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fda94030380 tx=0x7fda94032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:26.672 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.014+0000 7fda937fe640 1 -- 172.21.15.167:0/2176259123 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda94037040 con 0x7fdaa4105bb0 2024-06-18T12:57:26.672 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.014+0000 7fdaa8d08640 1 -- 172.21.15.167:0/2176259123 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa4199850 con 0x7fdaa4105bb0 2024-06-18T12:57:26.672 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.014+0000 7fda937fe640 1 -- 172.21.15.167:0/2176259123 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7fda94040070 con 0x7fdaa4105bb0 2024-06-18T12:57:26.672 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.014+0000 7fda937fe640 1 -- 172.21.15.167:0/2176259123 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda9403bc30 con 0x7fdaa4105bb0 2024-06-18T12:57:26.672 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.014+0000 7fdaa8d08640 1 -- 172.21.15.167:0/2176259123 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa419c540 con 0x7fdaa4105bb0 2024-06-18T12:57:26.672 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.014+0000 7fda937fe640 1 -- 172.21.15.167:0/2176259123 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fda94045460 con 0x7fdaa4105bb0 2024-06-18T12:57:26.672 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.014+0000 7fda937fe640 1 -- 172.21.15.167:0/2176259123 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fda94044680 con 0x7fdaa4105bb0 2024-06-18T12:57:26.673 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.015+0000 7fdaa8d08640 1 -- 172.21.15.167:0/2176259123 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa4106010 con 0x7fdaa4105bb0 2024-06-18T12:57:26.673 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.017+0000 7fda937fe640 1 -- 172.21.15.167:0/2176259123 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fda9403b430 con 0x7fdaa4105bb0 2024-06-18T12:57:26.673 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.082+0000 7fdaa8d08640 1 -- 172.21.15.167:0/2176259123 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdaa419c820 con 0x7fdaa4105bb0 2024-06-18T12:57:26.673 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.082+0000 7fda937fe640 1 -- 172.21.15.167:0/2176259123 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fda9403b610 con 0x7fdaa4105bb0 2024-06-18T12:57:26.673 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.083+0000 7fdaa8d08640 1 -- 172.21.15.167:0/2176259123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4105bb0 msgr2=0x7fdaa4199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:26.673 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.083+0000 7fdaa8d08640 1 --2- 172.21.15.167:0/2176259123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4105bb0 0x7fdaa4199080 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fda94030380 tx=0x7fda94032670 comp rx=0 tx=0).stop 2024-06-18T12:57:26.673 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.083+0000 7fdaa8d08640 1 -- 172.21.15.167:0/2176259123 shutdown_connections 2024-06-18T12:57:26.673 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.083+0000 7fdaa8d08640 1 --2- 172.21.15.167:0/2176259123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaa4105bb0 0x7fdaa4199080 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:26.673 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.083+0000 7fdaa8d08640 1 -- 172.21.15.167:0/2176259123 >> 172.21.15.167:0/2176259123 conn(0x7fdaa40ff790 msgr2=0x7fdaa4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:26.673 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.084+0000 7fdaa8d08640 1 -- 172.21.15.167:0/2176259123 shutdown_connections 2024-06-18T12:57:26.673 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:26.084+0000 7fdaa8d08640 1 -- 172.21.15.167:0/2176259123 wait complete. 2024-06-18T12:57:26.673 INFO:teuthology.orchestra.run.smithi167.stdout:mgr not available, waiting (1/15)... 2024-06-18T12:57:27.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:27 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:27.132+0000 7f2c4d96d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-18T12:57:27.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:27 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:27.250+0000 7f2c4d96d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-18T12:57:27.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:27 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:27.309+0000 7f2c4d96d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-18T12:57:27.807 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:27 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:27.527+0000 7f2c4d96d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-18T12:57:27.807 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:27 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:27.590+0000 7f2c4d96d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-18T12:57:27.807 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:27 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:27.658+0000 7f2c4d96d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-18T12:57:27.807 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:27 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:27.807+0000 7f2c4d96d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-18T12:57:28.555 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:28 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:28.233+0000 7f2c4d96d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-18T12:57:28.555 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:28 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:28.555+0000 7f2c4d96d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-18T12:57:28.877 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:28 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:28.623+0000 7f2c4d96d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-18T12:57:28.877 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:28 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:28.785+0000 7f2c4d96d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-18T12:57:29.133 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:29 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:29.120+0000 7f2c4d96d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-18T12:57:29.385 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1342516902' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-18T12:57:29.385 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:29 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:29.193+0000 7f2c4d96d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-18T12:57:29.698 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:29 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:29.385+0000 7f2c4d96d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-18T12:57:29.698 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:29 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:29.545+0000 7f2c4d96d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-18T12:57:29.698 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:29 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:29.612+0000 7f2c4d96d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-18T12:57:29.698 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:29 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:29.698+0000 7f2c4d96d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-18T12:57:29.710 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 2024-06-18T12:57:29.711 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout { 2024-06-18T12:57:29.711 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "fsid": "3434f1e2-2d72-11ef-bc9d-c7b262605968", 2024-06-18T12:57:29.711 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "health": { 2024-06-18T12:57:29.711 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-18T12:57:29.711 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-18T12:57:29.711 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-18T12:57:29.711 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:29.711 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-18T12:57:29.711 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-18T12:57:29.711 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 0 2024-06-18T12:57:29.711 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout ], 2024-06-18T12:57:29.711 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-18T12:57:29.712 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "a" 2024-06-18T12:57:29.712 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout ], 2024-06-18T12:57:29.712 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-06-18T12:57:29.712 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-18T12:57:29.712 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-18T12:57:29.712 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-18T12:57:29.712 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-18T12:57:29.712 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:29.712 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-18T12:57:29.712 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-18T12:57:29.712 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-18T12:57:29.712 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-18T12:57:29.712 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-18T12:57:29.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-18T12:57:29.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-18T12:57:29.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-18T12:57:29.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:29.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-18T12:57:29.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-18T12:57:29.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-18T12:57:29.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-18T12:57:29.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-18T12:57:29.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-18T12:57:29.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-18T12:57:29.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-18T12:57:29.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "restful" 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout ], 2024-06-18T12:57:29.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-18T12:57:29.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:29.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-18T12:57:29.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-18T12:57:29.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "modified": "2024-06-18T12:57:18.946357+0000", 2024-06-18T12:57:29.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-18T12:57:29.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:29.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-18T12:57:29.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout } 2024-06-18T12:57:29.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.062+0000 7fef0be85640 1 Processor -- start 2024-06-18T12:57:29.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.062+0000 7fef0be85640 1 -- start start 2024-06-18T12:57:29.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.063+0000 7fef0be85640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef04105b10 0x7fef04107f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:29.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.063+0000 7fef0be85640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fef040796c0 con 0x7fef04105b10 2024-06-18T12:57:29.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.063+0000 7fef09c21640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef04105b10 0x7fef04107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:29.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.063+0000 7fef09c21640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef04105b10 0x7fef04107f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46040/0 (socket says 172.21.15.167:46040) 2024-06-18T12:57:29.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.063+0000 7fef09c21640 1 -- 172.21.15.167:0/1484702380 learned_addr learned my addr 172.21.15.167:0/1484702380 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:29.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.063+0000 7fef09c21640 1 -- 172.21.15.167:0/1484702380 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef04108440 con 0x7fef04105b10 2024-06-18T12:57:29.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.064+0000 7fef09c21640 1 --2- 172.21.15.167:0/1484702380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef04105b10 0x7fef04107f00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fef00009820 tx=0x7fef0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e06af5b19acf097 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:29.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.064+0000 7fef08c1f640 1 -- 172.21.15.167:0/1484702380 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef00037070 con 0x7fef04105b10 2024-06-18T12:57:29.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.064+0000 7fef08c1f640 1 -- 172.21.15.167:0/1484702380 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7fef000328a0 con 0x7fef04105b10 2024-06-18T12:57:29.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.064+0000 7fef08c1f640 1 -- 172.21.15.167:0/1484702380 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef00030640 con 0x7fef04105b10 2024-06-18T12:57:29.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.065+0000 7fef0be85640 1 -- 172.21.15.167:0/1484702380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef04105b10 msgr2=0x7fef04107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:29.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.065+0000 7fef0be85640 1 --2- 172.21.15.167:0/1484702380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef04105b10 0x7fef04107f00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fef00009820 tx=0x7fef0002a270 comp rx=0 tx=0).stop 2024-06-18T12:57:29.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.065+0000 7fef0be85640 1 -- 172.21.15.167:0/1484702380 shutdown_connections 2024-06-18T12:57:29.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.065+0000 7fef0be85640 1 --2- 172.21.15.167:0/1484702380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef04105b10 0x7fef04107f00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:29.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.065+0000 7fef0be85640 1 -- 172.21.15.167:0/1484702380 >> 172.21.15.167:0/1484702380 conn(0x7fef040ff750 msgr2=0x7fef04101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:29.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.065+0000 7fef0be85640 1 -- 172.21.15.167:0/1484702380 shutdown_connections 2024-06-18T12:57:29.724 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.065+0000 7fef0be85640 1 -- 172.21.15.167:0/1484702380 wait complete. 2024-06-18T12:57:29.724 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.065+0000 7fef0be85640 1 Processor -- start 2024-06-18T12:57:29.724 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.065+0000 7fef0be85640 1 -- start start 2024-06-18T12:57:29.724 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.066+0000 7fef0be85640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef04105b10 0x7fef04198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:29.724 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.066+0000 7fef0be85640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fef041093c0 con 0x7fef04105b10 2024-06-18T12:57:29.724 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.066+0000 7fef09c21640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef04105b10 0x7fef04198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:29.724 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.066+0000 7fef09c21640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef04105b10 0x7fef04198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46050/0 (socket says 172.21.15.167:46050) 2024-06-18T12:57:29.724 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.066+0000 7fef09c21640 1 -- 172.21.15.167:0/1342516902 learned_addr learned my addr 172.21.15.167:0/1342516902 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:29.724 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.066+0000 7fef09c21640 1 -- 172.21.15.167:0/1342516902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef04199480 con 0x7fef04105b10 2024-06-18T12:57:29.724 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.066+0000 7fef09c21640 1 --2- 172.21.15.167:0/1342516902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef04105b10 0x7fef04198f40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef00000c00 tx=0x7fef000327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:29.724 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.066+0000 7feefaffd640 1 -- 172.21.15.167:0/1342516902 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef00037040 con 0x7fef04105b10 2024-06-18T12:57:29.724 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.066+0000 7fef0be85640 1 -- 172.21.15.167:0/1342516902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef04199710 con 0x7fef04105b10 2024-06-18T12:57:29.724 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.066+0000 7feefaffd640 1 -- 172.21.15.167:0/1342516902 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7fef00040070 con 0x7fef04105b10 2024-06-18T12:57:29.725 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.066+0000 7feefaffd640 1 -- 172.21.15.167:0/1342516902 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef0003bda0 con 0x7fef04105b10 2024-06-18T12:57:29.725 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.066+0000 7fef0be85640 1 -- 172.21.15.167:0/1342516902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0419c400 con 0x7fef04105b10 2024-06-18T12:57:29.725 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.067+0000 7feefaffd640 1 -- 172.21.15.167:0/1342516902 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fef00045460 con 0x7fef04105b10 2024-06-18T12:57:29.725 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.067+0000 7feefaffd640 1 -- 172.21.15.167:0/1342516902 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fef00044720 con 0x7fef04105b10 2024-06-18T12:57:29.725 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.067+0000 7fef0be85640 1 -- 172.21.15.167:0/1342516902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef04105f70 con 0x7fef04105b10 2024-06-18T12:57:29.725 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.069+0000 7feefaffd640 1 -- 172.21.15.167:0/1342516902 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fef0003b430 con 0x7fef04105b10 2024-06-18T12:57:29.725 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.133+0000 7fef0be85640 1 -- 172.21.15.167:0/1342516902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fef04199b90 con 0x7fef04105b10 2024-06-18T12:57:29.725 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.133+0000 7feefaffd640 1 -- 172.21.15.167:0/1342516902 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fef0003b6f0 con 0x7fef04105b10 2024-06-18T12:57:29.725 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.134+0000 7fef0be85640 1 -- 172.21.15.167:0/1342516902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef04105b10 msgr2=0x7fef04198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:29.725 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.134+0000 7fef0be85640 1 --2- 172.21.15.167:0/1342516902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef04105b10 0x7fef04198f40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef00000c00 tx=0x7fef000327b0 comp rx=0 tx=0).stop 2024-06-18T12:57:29.725 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.134+0000 7fef0be85640 1 -- 172.21.15.167:0/1342516902 shutdown_connections 2024-06-18T12:57:29.725 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.134+0000 7fef0be85640 1 --2- 172.21.15.167:0/1342516902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef04105b10 0x7fef04198f40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:29.725 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.134+0000 7fef0be85640 1 -- 172.21.15.167:0/1342516902 >> 172.21.15.167:0/1342516902 conn(0x7fef040ff750 msgr2=0x7fef04100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:29.725 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.135+0000 7fef0be85640 1 -- 172.21.15.167:0/1342516902 shutdown_connections 2024-06-18T12:57:29.726 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:29.135+0000 7fef0be85640 1 -- 172.21.15.167:0/1342516902 wait complete. 2024-06-18T12:57:29.726 INFO:teuthology.orchestra.run.smithi167.stdout:mgr not available, waiting (2/15)... 2024-06-18T12:57:29.976 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:29 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:29.762+0000 7f2c4d96d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-18T12:57:30.726 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:30.308+0000 7f2c4d96d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-18T12:57:30.726 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:30.470+0000 7f2c4d96d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-18T12:57:30.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-mon[29496]: Activating manager daemon y 2024-06-18T12:57:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-mon[29496]: mgrmap e2: y(active, starting, since 0.00303319s) 2024-06-18T12:57:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-mon[29496]: from='mgr.14100 172.21.15.167:0/1215181486' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-18T12:57:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-mon[29496]: from='mgr.14100 172.21.15.167:0/1215181486' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-18T12:57:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-mon[29496]: from='mgr.14100 172.21.15.167:0/1215181486' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-18T12:57:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-mon[29496]: from='mgr.14100 172.21.15.167:0/1215181486' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T12:57:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-mon[29496]: from='mgr.14100 172.21.15.167:0/1215181486' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-18T12:57:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-mon[29496]: Manager daemon y is now available 2024-06-18T12:57:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-mon[29496]: from='mgr.14100 172.21.15.167:0/1215181486' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-18T12:57:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-mon[29496]: from='mgr.14100 172.21.15.167:0/1215181486' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-18T12:57:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-mon[29496]: from='mgr.14100 172.21.15.167:0/1215181486' entity='mgr.y' 2024-06-18T12:57:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-mon[29496]: from='mgr.14100 172.21.15.167:0/1215181486' entity='mgr.y' 2024-06-18T12:57:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:30 smithi167 ceph-mon[29496]: from='mgr.14100 172.21.15.167:0/1215181486' entity='mgr.y' 2024-06-18T12:57:32.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:32 smithi167 ceph-mon[29496]: mgrmap e3: y(active, since 1.00709s) 2024-06-18T12:57:32.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2780627846' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-18T12:57:33.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:33 smithi167 ceph-mon[29496]: mgrmap e4: y(active, since 2s) 2024-06-18T12:57:34.474 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 2024-06-18T12:57:34.474 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout { 2024-06-18T12:57:34.475 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "fsid": "3434f1e2-2d72-11ef-bc9d-c7b262605968", 2024-06-18T12:57:34.475 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "health": { 2024-06-18T12:57:34.475 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-18T12:57:34.475 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-18T12:57:34.475 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-18T12:57:34.475 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:34.475 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-18T12:57:34.475 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-18T12:57:34.475 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 0 2024-06-18T12:57:34.475 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout ], 2024-06-18T12:57:34.475 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-18T12:57:34.475 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "a" 2024-06-18T12:57:34.475 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout ], 2024-06-18T12:57:34.476 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-06-18T12:57:34.476 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-18T12:57:34.476 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-18T12:57:34.476 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-18T12:57:34.476 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-18T12:57:34.476 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:34.476 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-18T12:57:34.476 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-18T12:57:34.476 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-18T12:57:34.476 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-18T12:57:34.476 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-18T12:57:34.484 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-18T12:57:34.484 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-18T12:57:34.484 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-18T12:57:34.485 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:34.485 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-18T12:57:34.485 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-18T12:57:34.485 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-18T12:57:34.485 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-18T12:57:34.485 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-18T12:57:34.485 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-18T12:57:34.485 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-18T12:57:34.485 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-18T12:57:34.485 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-18T12:57:34.485 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:34.485 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-18T12:57:34.485 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-18T12:57:34.486 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-18T12:57:34.486 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-18T12:57:34.486 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:34.486 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-18T12:57:34.486 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-18T12:57:34.486 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-18T12:57:34.486 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-18T12:57:34.486 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-18T12:57:34.486 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-18T12:57:34.486 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "restful" 2024-06-18T12:57:34.486 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout ], 2024-06-18T12:57:34.486 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-18T12:57:34.487 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:34.487 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-18T12:57:34.487 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-18T12:57:34.487 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "modified": "2024-06-18T12:57:18.946357+0000", 2024-06-18T12:57:34.487 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-18T12:57:34.487 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout }, 2024-06-18T12:57:34.487 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-18T12:57:34.487 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout } 2024-06-18T12:57:34.487 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.089+0000 7f928b3ad640 1 Processor -- start 2024-06-18T12:57:34.487 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.089+0000 7f928b3ad640 1 -- start start 2024-06-18T12:57:34.487 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.090+0000 7f928b3ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f928407b6b0 0x7f9284079b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:34.487 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.090+0000 7f928b3ad640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f928407a0e0 con 0x7f928407b6b0 2024-06-18T12:57:34.487 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.090+0000 7f9289149640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f928407b6b0 0x7f9284079b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:34.488 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.090+0000 7f9289149640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f928407b6b0 0x7f9284079b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46126/0 (socket says 172.21.15.167:46126) 2024-06-18T12:57:34.488 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.090+0000 7f9289149640 1 -- 172.21.15.167:0/4255938595 learned_addr learned my addr 172.21.15.167:0/4255938595 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:34.488 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.090+0000 7f9289149640 1 -- 172.21.15.167:0/4255938595 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928407a260 con 0x7f928407b6b0 2024-06-18T12:57:34.488 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.091+0000 7f9289149640 1 --2- 172.21.15.167:0/4255938595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f928407b6b0 0x7f9284079b10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9274009e80 tx=0x7f927402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88458dfefbf0acc server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:34.488 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.091+0000 7f9283fff640 1 -- 172.21.15.167:0/4255938595 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9274036070 con 0x7f928407b6b0 2024-06-18T12:57:34.488 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.091+0000 7f9283fff640 1 -- 172.21.15.167:0/4255938595 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f9274031860 con 0x7f928407b6b0 2024-06-18T12:57:34.488 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.091+0000 7f9283fff640 1 -- 172.21.15.167:0/4255938595 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92740077e0 con 0x7f928407b6b0 2024-06-18T12:57:34.488 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.092+0000 7f928b3ad640 1 -- 172.21.15.167:0/4255938595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f928407b6b0 msgr2=0x7f9284079b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:34.488 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.092+0000 7f928b3ad640 1 --2- 172.21.15.167:0/4255938595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f928407b6b0 0x7f9284079b10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9274009e80 tx=0x7f927402a7a0 comp rx=0 tx=0).stop 2024-06-18T12:57:34.488 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.092+0000 7f928b3ad640 1 -- 172.21.15.167:0/4255938595 shutdown_connections 2024-06-18T12:57:34.488 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.092+0000 7f928b3ad640 1 --2- 172.21.15.167:0/4255938595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f928407b6b0 0x7f9284079b10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:34.488 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.092+0000 7f928b3ad640 1 -- 172.21.15.167:0/4255938595 >> 172.21.15.167:0/4255938595 conn(0x7f92841008b0 msgr2=0x7f9284102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:34.488 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.092+0000 7f928b3ad640 1 -- 172.21.15.167:0/4255938595 shutdown_connections 2024-06-18T12:57:34.489 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.092+0000 7f928b3ad640 1 -- 172.21.15.167:0/4255938595 wait complete. 2024-06-18T12:57:34.489 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.092+0000 7f928b3ad640 1 Processor -- start 2024-06-18T12:57:34.489 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.092+0000 7f928b3ad640 1 -- start start 2024-06-18T12:57:34.489 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.092+0000 7f928b3ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f928407b6b0 0x7f928419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:34.489 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.092+0000 7f928b3ad640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f928410b610 con 0x7f928407b6b0 2024-06-18T12:57:34.489 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.093+0000 7f9289149640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f928407b6b0 0x7f928419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:34.489 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.093+0000 7f9289149640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f928407b6b0 0x7f928419b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46130/0 (socket says 172.21.15.167:46130) 2024-06-18T12:57:34.489 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.093+0000 7f9289149640 1 -- 172.21.15.167:0/2780627846 learned_addr learned my addr 172.21.15.167:0/2780627846 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:34.489 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.093+0000 7f9289149640 1 -- 172.21.15.167:0/2780627846 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928419b6c0 con 0x7f928407b6b0 2024-06-18T12:57:34.489 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.093+0000 7f9289149640 1 --2- 172.21.15.167:0/2780627846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f928407b6b0 0x7f928419b180 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9274008d70 tx=0x7f9274008da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:34.489 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.093+0000 7f92827fc640 1 -- 172.21.15.167:0/2780627846 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9274041020 con 0x7f928407b6b0 2024-06-18T12:57:34.489 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.093+0000 7f92827fc640 1 -- 172.21.15.167:0/2780627846 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f927403c8d0 con 0x7f928407b6b0 2024-06-18T12:57:34.490 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.093+0000 7f928b3ad640 1 -- 172.21.15.167:0/2780627846 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928419b950 con 0x7f928407b6b0 2024-06-18T12:57:34.490 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.093+0000 7f92827fc640 1 -- 172.21.15.167:0/2780627846 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9274036040 con 0x7f928407b6b0 2024-06-18T12:57:34.490 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.093+0000 7f928b3ad640 1 -- 172.21.15.167:0/2780627846 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f928419e640 con 0x7f928407b6b0 2024-06-18T12:57:34.490 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.094+0000 7f92827fc640 1 -- 172.21.15.167:0/2780627846 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f9274044020 con 0x7f928407b6b0 2024-06-18T12:57:34.490 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.094+0000 7f92827fc640 1 --2- 172.21.15.167:0/2780627846 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f926c038e10 0x7f926c03b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:34.490 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.094+0000 7f92827fc640 1 -- 172.21.15.167:0/2780627846 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f927406cf70 con 0x7f928407b6b0 2024-06-18T12:57:34.490 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.094+0000 7f9288948640 1 --2- 172.21.15.167:0/2780627846 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f926c038e10 0x7f926c03b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:34.490 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.094+0000 7f928b3ad640 1 -- 172.21.15.167:0/2780627846 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9284079b90 con 0x7f928407b6b0 2024-06-18T12:57:34.490 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.095+0000 7f9288948640 1 --2- 172.21.15.167:0/2780627846 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f926c038e10 0x7f926c03b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f92780098c0 tx=0x7f9278006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:34.490 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.099+0000 7f92827fc640 1 -- 172.21.15.167:0/2780627846 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f927404b020 con 0x7f928407b6b0 2024-06-18T12:57:34.490 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.340+0000 7f928b3ad640 1 -- 172.21.15.167:0/2780627846 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f928419e990 con 0x7f928407b6b0 2024-06-18T12:57:34.491 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.340+0000 7f92827fc640 1 -- 172.21.15.167:0/2780627846 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1238 (secure 0 0 0) 0x7f9274038df0 con 0x7f928407b6b0 2024-06-18T12:57:34.491 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.343+0000 7f928b3ad640 1 -- 172.21.15.167:0/2780627846 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f926c038e10 msgr2=0x7f926c03b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:34.491 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.343+0000 7f928b3ad640 1 --2- 172.21.15.167:0/2780627846 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f926c038e10 0x7f926c03b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f92780098c0 tx=0x7f9278006db0 comp rx=0 tx=0).stop 2024-06-18T12:57:34.491 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.343+0000 7f928b3ad640 1 -- 172.21.15.167:0/2780627846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f928407b6b0 msgr2=0x7f928419b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:34.491 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.343+0000 7f928b3ad640 1 --2- 172.21.15.167:0/2780627846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f928407b6b0 0x7f928419b180 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9274008d70 tx=0x7f9274008da0 comp rx=0 tx=0).stop 2024-06-18T12:57:34.491 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.343+0000 7f928b3ad640 1 -- 172.21.15.167:0/2780627846 shutdown_connections 2024-06-18T12:57:34.491 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.343+0000 7f928b3ad640 1 --2- 172.21.15.167:0/2780627846 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f926c038e10 0x7f926c03b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:34.491 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.343+0000 7f928b3ad640 1 --2- 172.21.15.167:0/2780627846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f928407b6b0 0x7f928419b180 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:34.491 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.343+0000 7f928b3ad640 1 -- 172.21.15.167:0/2780627846 >> 172.21.15.167:0/2780627846 conn(0x7f92841008b0 msgr2=0x7f9284101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:34.491 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.343+0000 7f928b3ad640 1 -- 172.21.15.167:0/2780627846 shutdown_connections 2024-06-18T12:57:34.491 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:32.343+0000 7f928b3ad640 1 -- 172.21.15.167:0/2780627846 wait complete. 2024-06-18T12:57:34.491 INFO:teuthology.orchestra.run.smithi167.stdout:mgr is available 2024-06-18T12:57:37.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2731388847' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-18T12:57:43.875 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 2024-06-18T12:57:43.875 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout [global] 2024-06-18T12:57:43.875 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout fsid = 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:57:43.875 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-18T12:57:43.876 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-18T12:57:43.876 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-18T12:57:43.876 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-18T12:57:43.876 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 2024-06-18T12:57:43.876 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-18T12:57:43.876 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-18T12:57:43.876 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 2024-06-18T12:57:43.876 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout [osd] 2024-06-18T12:57:43.876 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-18T12:57:43.876 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-18T12:57:43.876 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.457+0000 7fbb58c7e640 1 Processor -- start 2024-06-18T12:57:43.877 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.457+0000 7fbb58c7e640 1 -- start start 2024-06-18T12:57:43.877 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.457+0000 7fbb58c7e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb541050c0 0x7fbb541054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:43.877 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.457+0000 7fbb58c7e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbb54105a90 con 0x7fbb541050c0 2024-06-18T12:57:43.877 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.458+0000 7fbb5259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb541050c0 0x7fbb541054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:43.877 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.458+0000 7fbb5259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb541050c0 0x7fbb541054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38364/0 (socket says 172.21.15.167:38364) 2024-06-18T12:57:43.877 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.458+0000 7fbb5259c640 1 -- 172.21.15.167:0/2293437401 learned_addr learned my addr 172.21.15.167:0/2293437401 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:43.877 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.458+0000 7fbb5259c640 1 -- 172.21.15.167:0/2293437401 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb541062c0 con 0x7fbb541050c0 2024-06-18T12:57:43.877 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.458+0000 7fbb5259c640 1 --2- 172.21.15.167:0/2293437401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb541050c0 0x7fbb541054c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009820 tx=0x7fbb4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66445920095ed465 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:43.877 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.459+0000 7fbb5159a640 1 -- 172.21.15.167:0/2293437401 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb44037070 con 0x7fbb541050c0 2024-06-18T12:57:43.877 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.459+0000 7fbb5159a640 1 -- 172.21.15.167:0/2293437401 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7fbb440328a0 con 0x7fbb541050c0 2024-06-18T12:57:43.877 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.459+0000 7fbb5159a640 1 -- 172.21.15.167:0/2293437401 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb44030640 con 0x7fbb541050c0 2024-06-18T12:57:43.878 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.459+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2293437401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb541050c0 msgr2=0x7fbb541054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:43.878 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.459+0000 7fbb58c7e640 1 --2- 172.21.15.167:0/2293437401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb541050c0 0x7fbb541054c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009820 tx=0x7fbb4402a270 comp rx=0 tx=0).stop 2024-06-18T12:57:43.878 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.459+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2293437401 shutdown_connections 2024-06-18T12:57:43.878 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.459+0000 7fbb58c7e640 1 --2- 172.21.15.167:0/2293437401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb541050c0 0x7fbb541054c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:43.878 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.459+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2293437401 >> 172.21.15.167:0/2293437401 conn(0x7fbb541008b0 msgr2=0x7fbb54102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:43.878 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.460+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2293437401 shutdown_connections 2024-06-18T12:57:43.878 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.460+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2293437401 wait complete. 2024-06-18T12:57:43.878 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.460+0000 7fbb58c7e640 1 Processor -- start 2024-06-18T12:57:43.878 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.460+0000 7fbb58c7e640 1 -- start start 2024-06-18T12:57:43.878 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.461+0000 7fbb58c7e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb541050c0 0x7fbb54198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:43.878 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.461+0000 7fbb58c7e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbb541094f0 con 0x7fbb541050c0 2024-06-18T12:57:43.879 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.461+0000 7fbb5259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb541050c0 0x7fbb54198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:43.879 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.461+0000 7fbb5259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb541050c0 0x7fbb54198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38378/0 (socket says 172.21.15.167:38378) 2024-06-18T12:57:43.879 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.461+0000 7fbb5259c640 1 -- 172.21.15.167:0/2731388847 learned_addr learned my addr 172.21.15.167:0/2731388847 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:43.879 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.461+0000 7fbb5259c640 1 -- 172.21.15.167:0/2731388847 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb541994c0 con 0x7fbb541050c0 2024-06-18T12:57:43.879 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.461+0000 7fbb5259c640 1 --2- 172.21.15.167:0/2731388847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb541050c0 0x7fbb54198f80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbb44000c00 tx=0x7fbb440327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:43.879 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.461+0000 7fbb437fe640 1 -- 172.21.15.167:0/2731388847 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb44037040 con 0x7fbb541050c0 2024-06-18T12:57:43.879 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.461+0000 7fbb437fe640 1 -- 172.21.15.167:0/2731388847 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7fbb44040070 con 0x7fbb541050c0 2024-06-18T12:57:43.879 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.461+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2731388847 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb54199750 con 0x7fbb541050c0 2024-06-18T12:57:43.879 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.461+0000 7fbb437fe640 1 -- 172.21.15.167:0/2731388847 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb4403bda0 con 0x7fbb541050c0 2024-06-18T12:57:43.879 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.461+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2731388847 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb5419c440 con 0x7fbb541050c0 2024-06-18T12:57:43.879 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.462+0000 7fbb437fe640 1 -- 172.21.15.167:0/2731388847 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fbb4403b6f0 con 0x7fbb541050c0 2024-06-18T12:57:43.880 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.462+0000 7fbb437fe640 1 --2- 172.21.15.167:0/2731388847 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7fbb3c038f70 0x7fbb3c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:43.880 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.463+0000 7fbb437fe640 1 -- 172.21.15.167:0/2731388847 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbb4406bdf0 con 0x7fbb541050c0 2024-06-18T12:57:43.880 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.463+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2731388847 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb54105540 con 0x7fbb541050c0 2024-06-18T12:57:43.880 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.463+0000 7fbb51d9b640 1 --2- 172.21.15.167:0/2731388847 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7fbb3c038f70 0x7fbb3c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:43.880 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.463+0000 7fbb51d9b640 1 --2- 172.21.15.167:0/2731388847 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7fbb3c038f70 0x7fbb3c03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbb480098c0 tx=0x7fbb48006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:43.880 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.467+0000 7fbb437fe640 1 -- 172.21.15.167:0/2731388847 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fbb440371e0 con 0x7fbb541050c0 2024-06-18T12:57:43.880 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.647+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2731388847 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbb5419d9e0 con 0x7fbb541050c0 2024-06-18T12:57:43.880 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.648+0000 7fbb437fe640 1 -- 172.21.15.167:0/2731388847 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+282 (secure 0 0 0) 0x7fbb4403b430 con 0x7fbb541050c0 2024-06-18T12:57:43.880 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.650+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2731388847 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7fbb3c038f70 msgr2=0x7fbb3c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:43.880 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.650+0000 7fbb58c7e640 1 --2- 172.21.15.167:0/2731388847 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7fbb3c038f70 0x7fbb3c03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbb480098c0 tx=0x7fbb48006db0 comp rx=0 tx=0).stop 2024-06-18T12:57:43.880 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.650+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2731388847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb541050c0 msgr2=0x7fbb54198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:43.881 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.650+0000 7fbb58c7e640 1 --2- 172.21.15.167:0/2731388847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb541050c0 0x7fbb54198f80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbb44000c00 tx=0x7fbb440327b0 comp rx=0 tx=0).stop 2024-06-18T12:57:43.881 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.650+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2731388847 shutdown_connections 2024-06-18T12:57:43.881 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.650+0000 7fbb58c7e640 1 --2- 172.21.15.167:0/2731388847 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7fbb3c038f70 0x7fbb3c03b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:43.881 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.650+0000 7fbb58c7e640 1 --2- 172.21.15.167:0/2731388847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb541050c0 0x7fbb54198f80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:43.881 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.650+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2731388847 >> 172.21.15.167:0/2731388847 conn(0x7fbb541008b0 msgr2=0x7fbb54101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:43.881 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.650+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2731388847 shutdown_connections 2024-06-18T12:57:43.881 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:37.650+0000 7fbb58c7e640 1 -- 172.21.15.167:0/2731388847 wait complete. 2024-06-18T12:57:43.881 INFO:teuthology.orchestra.run.smithi167.stdout:Enabling cephadm module... 2024-06-18T12:57:46.434 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4072190815' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-18T12:57:46.435 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:46 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ignoring --setuser ceph since I am not root 2024-06-18T12:57:46.435 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:46 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ignoring --setgroup ceph since I am not root 2024-06-18T12:57:46.435 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:46 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:46.327+0000 7f393fa99040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-18T12:57:46.710 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:46 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:46.485+0000 7f393fa99040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-18T12:57:46.710 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:46 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:46.553+0000 7f393fa99040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-18T12:57:46.718 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.900+0000 7f95cec3d640 1 Processor -- start 2024-06-18T12:57:46.718 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.900+0000 7f95cec3d640 1 -- start start 2024-06-18T12:57:46.718 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.900+0000 7f95cec3d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c81050c0 0x7f95c81054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:46.718 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.900+0000 7f95cec3d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95c8105a90 con 0x7f95c81050c0 2024-06-18T12:57:46.719 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.901+0000 7f95cc9d9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c81050c0 0x7f95c81054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:46.719 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.901+0000 7f95cc9d9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c81050c0 0x7f95c81054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58272/0 (socket says 172.21.15.167:58272) 2024-06-18T12:57:46.719 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.901+0000 7f95cc9d9640 1 -- 172.21.15.167:0/3633139902 learned_addr learned my addr 172.21.15.167:0/3633139902 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:46.719 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.901+0000 7f95cc9d9640 1 -- 172.21.15.167:0/3633139902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c81062c0 con 0x7f95c81050c0 2024-06-18T12:57:46.719 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.902+0000 7f95cc9d9640 1 --2- 172.21.15.167:0/3633139902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c81050c0 0x7f95c81054c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009820 tx=0x7f95b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8367487bd96723 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:46.719 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.902+0000 7f95c77fe640 1 -- 172.21.15.167:0/3633139902 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95b8037070 con 0x7f95c81050c0 2024-06-18T12:57:46.719 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.902+0000 7f95c77fe640 1 -- 172.21.15.167:0/3633139902 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f95b80328a0 con 0x7f95c81050c0 2024-06-18T12:57:46.719 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.902+0000 7f95c77fe640 1 -- 172.21.15.167:0/3633139902 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95b8030640 con 0x7f95c81050c0 2024-06-18T12:57:46.719 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.902+0000 7f95cec3d640 1 -- 172.21.15.167:0/3633139902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c81050c0 msgr2=0x7f95c81054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:46.719 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.902+0000 7f95cec3d640 1 --2- 172.21.15.167:0/3633139902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c81050c0 0x7f95c81054c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009820 tx=0x7f95b802a270 comp rx=0 tx=0).stop 2024-06-18T12:57:46.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.903+0000 7f95cec3d640 1 -- 172.21.15.167:0/3633139902 shutdown_connections 2024-06-18T12:57:46.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.903+0000 7f95cec3d640 1 --2- 172.21.15.167:0/3633139902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c81050c0 0x7f95c81054c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:46.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.903+0000 7f95cec3d640 1 -- 172.21.15.167:0/3633139902 >> 172.21.15.167:0/3633139902 conn(0x7f95c81008b0 msgr2=0x7f95c8102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:46.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.903+0000 7f95cec3d640 1 -- 172.21.15.167:0/3633139902 shutdown_connections 2024-06-18T12:57:46.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.903+0000 7f95cec3d640 1 -- 172.21.15.167:0/3633139902 wait complete. 2024-06-18T12:57:46.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.903+0000 7f95cec3d640 1 Processor -- start 2024-06-18T12:57:46.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.903+0000 7f95cec3d640 1 -- start start 2024-06-18T12:57:46.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.903+0000 7f95cec3d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c81050c0 0x7f95c807b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:46.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.903+0000 7f95cec3d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81094f0 con 0x7f95c81050c0 2024-06-18T12:57:46.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.904+0000 7f95cc9d9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c81050c0 0x7f95c807b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:46.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.904+0000 7f95cc9d9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c81050c0 0x7f95c807b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58282/0 (socket says 172.21.15.167:58282) 2024-06-18T12:57:46.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.904+0000 7f95cc9d9640 1 -- 172.21.15.167:0/4072190815 learned_addr learned my addr 172.21.15.167:0/4072190815 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:46.720 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.904+0000 7f95cc9d9640 1 -- 172.21.15.167:0/4072190815 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c807b8c0 con 0x7f95c81050c0 2024-06-18T12:57:46.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.904+0000 7f95cc9d9640 1 --2- 172.21.15.167:0/4072190815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c81050c0 0x7f95c807b380 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f95b8000c00 tx=0x7f95b80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:46.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.904+0000 7f95c5ffb640 1 -- 172.21.15.167:0/4072190815 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95b8037040 con 0x7f95c81050c0 2024-06-18T12:57:46.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.904+0000 7f95c5ffb640 1 -- 172.21.15.167:0/4072190815 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f95b8040070 con 0x7f95c81050c0 2024-06-18T12:57:46.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.904+0000 7f95c5ffb640 1 -- 172.21.15.167:0/4072190815 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95b803bda0 con 0x7f95c81050c0 2024-06-18T12:57:46.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.904+0000 7f95cec3d640 1 -- 172.21.15.167:0/4072190815 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c8079ab0 con 0x7f95c81050c0 2024-06-18T12:57:46.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.904+0000 7f95cec3d640 1 -- 172.21.15.167:0/4072190815 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c8079f90 con 0x7f95c81050c0 2024-06-18T12:57:46.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.905+0000 7f95c5ffb640 1 -- 172.21.15.167:0/4072190815 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f95b803b6f0 con 0x7f95c81050c0 2024-06-18T12:57:46.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.905+0000 7f95cec3d640 1 -- 172.21.15.167:0/4072190815 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c8105540 con 0x7f95c81050c0 2024-06-18T12:57:46.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.905+0000 7f95c5ffb640 1 --2- 172.21.15.167:0/4072190815 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f95b0038f20 0x7f95b003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:46.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.906+0000 7f95c5ffb640 1 -- 172.21.15.167:0/4072190815 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f95b806be20 con 0x7f95c81050c0 2024-06-18T12:57:46.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.906+0000 7f95c7fff640 1 --2- 172.21.15.167:0/4072190815 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f95b0038f20 0x7f95b003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:46.721 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.906+0000 7f95c7fff640 1 --2- 172.21.15.167:0/4072190815 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f95b0038f20 0x7f95b003b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0098c0 tx=0x7f95bc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:46.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:45.910+0000 7f95c5ffb640 1 -- 172.21.15.167:0/4072190815 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f95b80371e0 con 0x7f95c81050c0 2024-06-18T12:57:46.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:46.131+0000 7f95cec3d640 1 -- 172.21.15.167:0/4072190815 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f95c807a5b0 con 0x7f95c81050c0 2024-06-18T12:57:46.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:46.183+0000 7f95c5ffb640 1 -- 172.21.15.167:0/4072190815 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f95b806bb80 con 0x7f95c81050c0 2024-06-18T12:57:46.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:46.184+0000 7f95c5ffb640 1 -- 172.21.15.167:0/4072190815 <== mon.0 v2:172.21.15.167:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f95b8071020 con 0x7f95c81050c0 2024-06-18T12:57:46.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:46.186+0000 7f95cec3d640 1 -- 172.21.15.167:0/4072190815 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f95b0038f20 msgr2=0x7f95b003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:46.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:46.186+0000 7f95cec3d640 1 --2- 172.21.15.167:0/4072190815 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f95b0038f20 0x7f95b003b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0098c0 tx=0x7f95bc006db0 comp rx=0 tx=0).stop 2024-06-18T12:57:46.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:46.186+0000 7f95cec3d640 1 -- 172.21.15.167:0/4072190815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c81050c0 msgr2=0x7f95c807b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:46.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:46.186+0000 7f95cec3d640 1 --2- 172.21.15.167:0/4072190815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c81050c0 0x7f95c807b380 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f95b8000c00 tx=0x7f95b80327b0 comp rx=0 tx=0).stop 2024-06-18T12:57:46.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:46.186+0000 7f95cec3d640 1 -- 172.21.15.167:0/4072190815 shutdown_connections 2024-06-18T12:57:46.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:46.186+0000 7f95cec3d640 1 --2- 172.21.15.167:0/4072190815 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f95b0038f20 0x7f95b003b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:46.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:46.186+0000 7f95cec3d640 1 --2- 172.21.15.167:0/4072190815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c81050c0 0x7f95c807b380 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:46.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:46.186+0000 7f95cec3d640 1 -- 172.21.15.167:0/4072190815 >> 172.21.15.167:0/4072190815 conn(0x7f95c81008b0 msgr2=0x7f95c8101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:46.722 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:46.186+0000 7f95cec3d640 1 -- 172.21.15.167:0/4072190815 shutdown_connections 2024-06-18T12:57:46.723 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:46.186+0000 7f95cec3d640 1 -- 172.21.15.167:0/4072190815 wait complete. 2024-06-18T12:57:46.970 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:46 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:46.804+0000 7f393fa99040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-18T12:57:46.970 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:46 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:46.865+0000 7f393fa99040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-18T12:57:47.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4072190815' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-18T12:57:47.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:47 smithi167 ceph-mon[29496]: mgrmap e5: y(active, since 15s) 2024-06-18T12:57:47.925 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:47 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:47.832+0000 7f393fa99040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-18T12:57:47.934 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout { 2024-06-18T12:57:47.934 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-18T12:57:47.934 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-18T12:57:47.934 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-06-18T12:57:47.935 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-18T12:57:47.935 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout } 2024-06-18T12:57:47.935 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.134+0000 7f9272570640 1 Processor -- start 2024-06-18T12:57:47.935 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.134+0000 7f9272570640 1 -- start start 2024-06-18T12:57:47.935 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.135+0000 7f9272570640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f926c104040 0x7f926c104440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:47.935 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.135+0000 7f9272570640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f926c104a10 con 0x7f926c104040 2024-06-18T12:57:47.935 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.135+0000 7f926bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f926c104040 0x7f926c104440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:47.935 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.135+0000 7f926bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f926c104040 0x7f926c104440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58302/0 (socket says 172.21.15.167:58302) 2024-06-18T12:57:47.935 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.135+0000 7f926bfff640 1 -- 172.21.15.167:0/2991562319 learned_addr learned my addr 172.21.15.167:0/2991562319 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:47.935 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.135+0000 7f926bfff640 1 -- 172.21.15.167:0/2991562319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926c105240 con 0x7f926c104040 2024-06-18T12:57:47.935 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.136+0000 7f926bfff640 1 --2- 172.21.15.167:0/2991562319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f926c104040 0x7f926c104440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f925c009820 tx=0x7f925c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be638e61fac2d77 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:47.935 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.136+0000 7f926b7fe640 1 -- 172.21.15.167:0/2991562319 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f925c037070 con 0x7f926c104040 2024-06-18T12:57:47.935 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.136+0000 7f926b7fe640 1 -- 172.21.15.167:0/2991562319 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f925c0328a0 con 0x7f926c104040 2024-06-18T12:57:47.936 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.136+0000 7f926b7fe640 1 -- 172.21.15.167:0/2991562319 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f925c030640 con 0x7f926c104040 2024-06-18T12:57:47.936 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.137+0000 7f9272570640 1 -- 172.21.15.167:0/2991562319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f926c104040 msgr2=0x7f926c104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:47.936 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.137+0000 7f9272570640 1 --2- 172.21.15.167:0/2991562319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f926c104040 0x7f926c104440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f925c009820 tx=0x7f925c02a270 comp rx=0 tx=0).stop 2024-06-18T12:57:47.936 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.137+0000 7f9272570640 1 -- 172.21.15.167:0/2991562319 shutdown_connections 2024-06-18T12:57:47.936 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.137+0000 7f9272570640 1 --2- 172.21.15.167:0/2991562319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f926c104040 0x7f926c104440 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:47.936 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.137+0000 7f9272570640 1 -- 172.21.15.167:0/2991562319 >> 172.21.15.167:0/2991562319 conn(0x7f926c0ff790 msgr2=0x7f926c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:47.936 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.137+0000 7f9272570640 1 -- 172.21.15.167:0/2991562319 shutdown_connections 2024-06-18T12:57:47.936 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.137+0000 7f9272570640 1 -- 172.21.15.167:0/2991562319 wait complete. 2024-06-18T12:57:47.936 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.137+0000 7f9272570640 1 Processor -- start 2024-06-18T12:57:47.936 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.137+0000 7f9272570640 1 -- start start 2024-06-18T12:57:47.936 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.138+0000 7f9272570640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f926c104040 0x7f926c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:47.936 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.138+0000 7f9272570640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f926c108470 con 0x7f926c104040 2024-06-18T12:57:47.937 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.138+0000 7f926bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f926c104040 0x7f926c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:47.937 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.138+0000 7f926bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f926c104040 0x7f926c199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58308/0 (socket says 172.21.15.167:58308) 2024-06-18T12:57:47.937 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.138+0000 7f926bfff640 1 -- 172.21.15.167:0/2329985159 learned_addr learned my addr 172.21.15.167:0/2329985159 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:47.937 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.138+0000 7f926bfff640 1 -- 172.21.15.167:0/2329985159 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926c199660 con 0x7f926c104040 2024-06-18T12:57:47.937 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.138+0000 7f926bfff640 1 --2- 172.21.15.167:0/2329985159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f926c104040 0x7f926c199120 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f925c030380 tx=0x7f925c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:47.937 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.138+0000 7f9269ffb640 1 -- 172.21.15.167:0/2329985159 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f925c037040 con 0x7f926c104040 2024-06-18T12:57:47.937 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.138+0000 7f9269ffb640 1 -- 172.21.15.167:0/2329985159 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f925c040070 con 0x7f926c104040 2024-06-18T12:57:47.937 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.138+0000 7f9272570640 1 -- 172.21.15.167:0/2329985159 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926c1998f0 con 0x7f926c104040 2024-06-18T12:57:47.937 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.138+0000 7f9269ffb640 1 -- 172.21.15.167:0/2329985159 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f925c03bc00 con 0x7f926c104040 2024-06-18T12:57:47.937 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.138+0000 7f9272570640 1 -- 172.21.15.167:0/2329985159 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f926c19c5e0 con 0x7f926c104040 2024-06-18T12:57:47.937 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.139+0000 7f9269ffb640 1 -- 172.21.15.167:0/2329985159 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f925c045460 con 0x7f926c104040 2024-06-18T12:57:47.937 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.139+0000 7f9272570640 1 -- 172.21.15.167:0/2329985159 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f926c1044c0 con 0x7f926c104040 2024-06-18T12:57:47.937 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.140+0000 7f9269ffb640 1 --2- 172.21.15.167:0/2329985159 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f9258038fc0 0x7f925803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:47.937 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.140+0000 7f9269ffb640 1 -- 172.21.15.167:0/2329985159 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f925c06bdf0 con 0x7f926c104040 2024-06-18T12:57:47.938 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.140+0000 7f9263fff640 1 -- 172.21.15.167:0/2329985159 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f9258038fc0 msgr2=0x7f925803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/2756279078 2024-06-18T12:57:47.938 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.140+0000 7f9263fff640 1 --2- 172.21.15.167:0/2329985159 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f9258038fc0 0x7f925803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-18T12:57:47.938 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.145+0000 7f9269ffb640 1 -- 172.21.15.167:0/2329985159 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f925c02e200 con 0x7f926c104040 2024-06-18T12:57:47.938 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.341+0000 7f9263fff640 1 -- 172.21.15.167:0/2329985159 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f9258038fc0 msgr2=0x7f925803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/2756279078 2024-06-18T12:57:47.938 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.341+0000 7f9263fff640 1 --2- 172.21.15.167:0/2329985159 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f9258038fc0 0x7f925803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-18T12:57:47.938 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.375+0000 7f9272570640 1 -- 172.21.15.167:0/2329985159 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f926c02d2e0 con 0x7f926c104040 2024-06-18T12:57:47.938 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.375+0000 7f9269ffb640 1 -- 172.21.15.167:0/2329985159 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f925c043520 con 0x7f926c104040 2024-06-18T12:57:47.938 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.377+0000 7f9272570640 1 -- 172.21.15.167:0/2329985159 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f9258038fc0 msgr2=0x7f925803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-18T12:57:47.938 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.377+0000 7f9272570640 1 --2- 172.21.15.167:0/2329985159 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f9258038fc0 0x7f925803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:47.938 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.377+0000 7f9272570640 1 -- 172.21.15.167:0/2329985159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f926c104040 msgr2=0x7f926c199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:47.938 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.377+0000 7f9272570640 1 --2- 172.21.15.167:0/2329985159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f926c104040 0x7f926c199120 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f925c030380 tx=0x7f925c032670 comp rx=0 tx=0).stop 2024-06-18T12:57:47.938 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.377+0000 7f9272570640 1 -- 172.21.15.167:0/2329985159 shutdown_connections 2024-06-18T12:57:47.939 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.377+0000 7f9272570640 1 --2- 172.21.15.167:0/2329985159 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f9258038fc0 0x7f925803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:47.939 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.377+0000 7f9272570640 1 --2- 172.21.15.167:0/2329985159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f926c104040 0x7f926c199120 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:47.939 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.377+0000 7f9272570640 1 -- 172.21.15.167:0/2329985159 >> 172.21.15.167:0/2329985159 conn(0x7f926c0ff790 msgr2=0x7f926c1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:47.939 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.377+0000 7f9272570640 1 -- 172.21.15.167:0/2329985159 shutdown_connections 2024-06-18T12:57:47.939 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:47.377+0000 7f9272570640 1 -- 172.21.15.167:0/2329985159 wait complete. 2024-06-18T12:57:47.939 INFO:teuthology.orchestra.run.smithi167.stdout:Waiting for the mgr to restart... 2024-06-18T12:57:47.939 INFO:teuthology.orchestra.run.smithi167.stdout:Waiting for mgr epoch 5... 2024-06-18T12:57:48.187 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:47 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:47.956+0000 7f393fa99040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-18T12:57:48.187 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:48 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:48.016+0000 7f393fa99040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-18T12:57:48.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2329985159' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-18T12:57:48.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:48 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:48.242+0000 7f393fa99040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-18T12:57:48.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:48 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:48.308+0000 7f393fa99040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-18T12:57:48.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:48 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:48.380+0000 7f393fa99040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-18T12:57:48.949 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:48 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:48.531+0000 7f393fa99040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-18T12:57:49.226 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:48 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:48.956+0000 7f393fa99040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-18T12:57:49.726 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:49 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:49.280+0000 7f393fa99040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-18T12:57:49.726 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:49 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:49.346+0000 7f393fa99040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-18T12:57:49.726 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:49 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:49.496+0000 7f393fa99040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-18T12:57:50.060 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:49 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:49.812+0000 7f393fa99040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-18T12:57:50.060 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:49 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:49.874+0000 7f393fa99040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-18T12:57:50.358 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:50 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:50.060+0000 7f393fa99040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-18T12:57:50.358 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:50 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:50.209+0000 7f393fa99040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-18T12:57:50.358 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:50 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:50.274+0000 7f393fa99040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-18T12:57:50.358 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:50 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:50.358+0000 7f393fa99040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-18T12:57:50.726 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:50 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:50.419+0000 7f393fa99040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-18T12:57:51.226 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:50 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:50.961+0000 7f393fa99040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-18T12:57:51.226 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:51.123+0000 7f393fa99040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-18T12:57:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: Active manager daemon y restarted 2024-06-18T12:57:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: Activating manager daemon y 2024-06-18T12:57:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: osdmap e2: 0 total, 0 up, 0 in 2024-06-18T12:57:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: mgrmap e6: y(active, starting, since 0.00333633s) 2024-06-18T12:57:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T12:57:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-18T12:57:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-18T12:57:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-18T12:57:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-18T12:57:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: Manager daemon y is now available 2024-06-18T12:57:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' 2024-06-18T12:57:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' 2024-06-18T12:57:51.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:57:51.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:57:51.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-18T12:57:51.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:51 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-18T12:57:52.440 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:52 smithi167 ceph-mon[29496]: Found migration_current of "None". Setting to last migration. 2024-06-18T12:57:52.441 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:52 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' 2024-06-18T12:57:52.441 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:52 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' 2024-06-18T12:57:52.441 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:52 smithi167 ceph-mon[29496]: mgrmap e7: y(active, since 1.00826s) 2024-06-18T12:57:52.759 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout { 2024-06-18T12:57:52.759 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-18T12:57:52.759 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-18T12:57:52.759 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout } 2024-06-18T12:57:52.759 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.342+0000 7f32b1bb4640 1 Processor -- start 2024-06-18T12:57:52.759 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.342+0000 7f32b1bb4640 1 -- start start 2024-06-18T12:57:52.759 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.342+0000 7f32b1bb4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32ac1050b0 0x7f32ac1054b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:52.759 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.342+0000 7f32b1bb4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac105a80 con 0x7f32ac1050b0 2024-06-18T12:57:52.759 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.343+0000 7f32ab7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32ac1050b0 0x7f32ac1054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:52.760 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.343+0000 7f32ab7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32ac1050b0 0x7f32ac1054b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58322/0 (socket says 172.21.15.167:58322) 2024-06-18T12:57:52.760 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.343+0000 7f32ab7fe640 1 -- 172.21.15.167:0/1356416693 learned_addr learned my addr 172.21.15.167:0/1356416693 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:52.760 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.343+0000 7f32ab7fe640 1 -- 172.21.15.167:0/1356416693 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac1062b0 con 0x7f32ac1050b0 2024-06-18T12:57:52.760 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.344+0000 7f32ab7fe640 1 --2- 172.21.15.167:0/1356416693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32ac1050b0 0x7f32ac1054b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3294009820 tx=0x7f329402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48234b2569a939ef server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:52.760 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.344+0000 7f32aa7fc640 1 -- 172.21.15.167:0/1356416693 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3294037070 con 0x7f32ac1050b0 2024-06-18T12:57:52.760 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.344+0000 7f32aa7fc640 1 -- 172.21.15.167:0/1356416693 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f32940328a0 con 0x7f32ac1050b0 2024-06-18T12:57:52.760 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.344+0000 7f32aa7fc640 1 -- 172.21.15.167:0/1356416693 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3294030640 con 0x7f32ac1050b0 2024-06-18T12:57:52.760 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.344+0000 7f32b1bb4640 1 -- 172.21.15.167:0/1356416693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32ac1050b0 msgr2=0x7f32ac1054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:52.760 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.344+0000 7f32b1bb4640 1 --2- 172.21.15.167:0/1356416693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32ac1050b0 0x7f32ac1054b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3294009820 tx=0x7f329402a270 comp rx=0 tx=0).stop 2024-06-18T12:57:52.760 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.345+0000 7f32b1bb4640 1 -- 172.21.15.167:0/1356416693 shutdown_connections 2024-06-18T12:57:52.760 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.345+0000 7f32b1bb4640 1 --2- 172.21.15.167:0/1356416693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32ac1050b0 0x7f32ac1054b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:52.760 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.345+0000 7f32b1bb4640 1 -- 172.21.15.167:0/1356416693 >> 172.21.15.167:0/1356416693 conn(0x7f32ac100800 msgr2=0x7f32ac102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:52.760 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.345+0000 7f32b1bb4640 1 -- 172.21.15.167:0/1356416693 shutdown_connections 2024-06-18T12:57:52.760 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.345+0000 7f32b1bb4640 1 -- 172.21.15.167:0/1356416693 wait complete. 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.345+0000 7f32b1bb4640 1 Processor -- start 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.345+0000 7f32b1bb4640 1 -- start start 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.346+0000 7f32b1bb4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32ac1050b0 0x7f32ac198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.346+0000 7f32b1bb4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac1094e0 con 0x7f32ac1050b0 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.346+0000 7f32ab7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32ac1050b0 0x7f32ac198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.346+0000 7f32ab7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32ac1050b0 0x7f32ac198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58328/0 (socket says 172.21.15.167:58328) 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.346+0000 7f32ab7fe640 1 -- 172.21.15.167:0/2170905477 learned_addr learned my addr 172.21.15.167:0/2170905477 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.346+0000 7f32ab7fe640 1 -- 172.21.15.167:0/2170905477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac1994d0 con 0x7f32ac1050b0 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.346+0000 7f32ab7fe640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32ac1050b0 0x7f32ac198f90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3294030380 tx=0x7f32940326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.346+0000 7f32a8ff9640 1 -- 172.21.15.167:0/2170905477 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3294037040 con 0x7f32ac1050b0 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.346+0000 7f32a8ff9640 1 -- 172.21.15.167:0/2170905477 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f3294040070 con 0x7f32ac1050b0 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.346+0000 7f32b1bb4640 1 -- 172.21.15.167:0/2170905477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac199760 con 0x7f32ac1050b0 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.346+0000 7f32a8ff9640 1 -- 172.21.15.167:0/2170905477 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f329403bc70 con 0x7f32ac1050b0 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.347+0000 7f32b1bb4640 1 -- 172.21.15.167:0/2170905477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac19c450 con 0x7f32ac1050b0 2024-06-18T12:57:52.761 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.348+0000 7f32a8ff9640 1 -- 172.21.15.167:0/2170905477 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f329403b430 con 0x7f32ac1050b0 2024-06-18T12:57:52.762 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.348+0000 7f32a8ff9640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f3298038fc0 0x7f329803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:52.762 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.348+0000 7f32a8ff9640 1 -- 172.21.15.167:0/2170905477 --> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f329803bb90 con 0x7f3298038fc0 2024-06-18T12:57:52.762 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.348+0000 7f32a8ff9640 1 -- 172.21.15.167:0/2170905477 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f329406beb0 con 0x7f32ac1050b0 2024-06-18T12:57:52.762 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.348+0000 7f32aaffd640 1 -- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f3298038fc0 msgr2=0x7f329803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/2756279078 2024-06-18T12:57:52.762 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.348+0000 7f32aaffd640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f3298038fc0 0x7f329803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-18T12:57:52.762 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.549+0000 7f32aaffd640 1 -- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f3298038fc0 msgr2=0x7f329803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/2756279078 2024-06-18T12:57:52.762 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.549+0000 7f32aaffd640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f3298038fc0 0x7f329803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-18T12:57:52.762 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.950+0000 7f32aaffd640 1 -- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f3298038fc0 msgr2=0x7f329803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/2756279078 2024-06-18T12:57:52.762 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:48.950+0000 7f32aaffd640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f3298038fc0 0x7f329803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-18T12:57:52.762 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:49.751+0000 7f32aaffd640 1 -- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f3298038fc0 msgr2=0x7f329803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/2756279078 2024-06-18T12:57:52.762 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:49.751+0000 7f32aaffd640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f3298038fc0 0x7f329803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-18T12:57:52.762 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:51.127+0000 7f32a8ff9640 1 -- 172.21.15.167:0/2170905477 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f329406acb0 con 0x7f32ac1050b0 2024-06-18T12:57:52.763 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:51.127+0000 7f32a8ff9640 1 -- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f3298038fc0 msgr2=0x7f329803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-18T12:57:52.763 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:51.127+0000 7f32a8ff9640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f3298038fc0 0x7f329803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:52.763 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.130+0000 7f32a8ff9640 1 -- 172.21.15.167:0/2170905477 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f3294030ac0 con 0x7f32ac1050b0 2024-06-18T12:57:52.763 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.130+0000 7f32a8ff9640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f329803c8f0 0x7f329803ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:52.763 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.130+0000 7f32a8ff9640 1 -- 172.21.15.167:0/2170905477 --> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f329403de70 con 0x7f329803c8f0 2024-06-18T12:57:52.763 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.131+0000 7f32aaffd640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f329803c8f0 0x7f329803ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:52.763 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.132+0000 7f32aaffd640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f329803c8f0 0x7f329803ece0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f329c003980 tx=0x7f329c0092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:52.763 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.133+0000 7f32a8ff9640 1 -- 172.21.15.167:0/2170905477 <== mgr.14118 v2:172.21.15.167:6800/1873453520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f329403de70 con 0x7f329803c8f0 2024-06-18T12:57:52.763 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.143+0000 7f32b1bb4640 1 -- 172.21.15.167:0/2170905477 --> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f32ac105530 con 0x7f329803c8f0 2024-06-18T12:57:52.763 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.143+0000 7f32a8ff9640 1 -- 172.21.15.167:0/2170905477 <== mgr.14118 v2:172.21.15.167:6800/1873453520 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f32ac105530 con 0x7f329803c8f0 2024-06-18T12:57:52.763 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.143+0000 7f32b1bb4640 1 -- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f329803c8f0 msgr2=0x7f329803ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:52.763 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.143+0000 7f32b1bb4640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f329803c8f0 0x7f329803ece0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f329c003980 tx=0x7f329c0092b0 comp rx=0 tx=0).stop 2024-06-18T12:57:52.764 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.143+0000 7f32b1bb4640 1 -- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32ac1050b0 msgr2=0x7f32ac198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:52.764 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.143+0000 7f32b1bb4640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32ac1050b0 0x7f32ac198f90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3294030380 tx=0x7f32940326e0 comp rx=0 tx=0).stop 2024-06-18T12:57:52.764 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.143+0000 7f32b1bb4640 1 -- 172.21.15.167:0/2170905477 shutdown_connections 2024-06-18T12:57:52.764 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.143+0000 7f32b1bb4640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/2756279078,v1:172.21.15.167:6801/2756279078] conn(0x7f3298038fc0 0x7f329803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:52.764 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.143+0000 7f32b1bb4640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f329803c8f0 0x7f329803ece0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:52.764 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.143+0000 7f32b1bb4640 1 --2- 172.21.15.167:0/2170905477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32ac1050b0 0x7f32ac198f90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:52.764 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.143+0000 7f32b1bb4640 1 -- 172.21.15.167:0/2170905477 >> 172.21.15.167:0/2170905477 conn(0x7f32ac100800 msgr2=0x7f32ac1013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:52.764 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.144+0000 7f32b1bb4640 1 -- 172.21.15.167:0/2170905477 shutdown_connections 2024-06-18T12:57:52.764 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:52.144+0000 7f32b1bb4640 1 -- 172.21.15.167:0/2170905477 wait complete. 2024-06-18T12:57:52.764 INFO:teuthology.orchestra.run.smithi167.stdout:mgr epoch 5 is available 2024-06-18T12:57:52.764 INFO:teuthology.orchestra.run.smithi167.stdout:Setting orchestrator backend to cephadm... 2024-06-18T12:57:53.344 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-mon[29496]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-18T12:57:53.345 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-mon[29496]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-18T12:57:53.345 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:53.202+0000 7f38ad0fd640 -1 log_channel(cephadm) log [ERR] : [18/Jun/2024:12:57:53] ENGINE Error in HTTPServer.serve 2024-06-18T12:57:53.345 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: Traceback (most recent call last): 2024-06-18T12:57:53.345 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-18T12:57:53.345 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: self._connections.run(self.expiration_interval) 2024-06-18T12:57:53.345 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-18T12:57:53.345 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: self._run(expiration_interval) 2024-06-18T12:57:53.345 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-18T12:57:53.345 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: new_conn = self._from_server_socket(self.server.socket) 2024-06-18T12:57:53.345 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-18T12:57:53.345 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-18T12:57:53.345 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-18T12:57:53.346 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: s = self.context.wrap_socket( 2024-06-18T12:57:53.346 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-18T12:57:53.346 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: return self.sslsocket_class._create( 2024-06-18T12:57:53.346 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-18T12:57:53.346 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: self.do_handshake() 2024-06-18T12:57:53.346 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-18T12:57:53.346 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: self._sslobj.do_handshake() 2024-06-18T12:57:53.346 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-18T12:57:53.346 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:53 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:57:53.881 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.140+0000 7f8c90557640 1 Processor -- start 2024-06-18T12:57:53.881 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.141+0000 7f8c90557640 1 -- start start 2024-06-18T12:57:53.881 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.141+0000 7f8c90557640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c88105ba0 0x7f8c88107f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:53.881 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.141+0000 7f8c90557640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c88079680 con 0x7f8c88105ba0 2024-06-18T12:57:53.881 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.141+0000 7f8c8e2f3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c88105ba0 0x7f8c88107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:53.882 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.141+0000 7f8c8e2f3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c88105ba0 0x7f8c88107f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60262/0 (socket says 172.21.15.167:60262) 2024-06-18T12:57:53.882 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.141+0000 7f8c8e2f3640 1 -- 172.21.15.167:0/3726515570 learned_addr learned my addr 172.21.15.167:0/3726515570 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:53.882 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.141+0000 7f8c8e2f3640 1 -- 172.21.15.167:0/3726515570 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c881084d0 con 0x7f8c88105ba0 2024-06-18T12:57:53.882 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.142+0000 7f8c8e2f3640 1 --2- 172.21.15.167:0/3726515570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c88105ba0 0x7f8c88107f90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009820 tx=0x7f8c8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd8a1762f40cfac server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:53.882 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.142+0000 7f8c8d2f1640 1 -- 172.21.15.167:0/3726515570 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c80037070 con 0x7f8c88105ba0 2024-06-18T12:57:53.882 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.142+0000 7f8c8d2f1640 1 -- 172.21.15.167:0/3726515570 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f8c800328a0 con 0x7f8c88105ba0 2024-06-18T12:57:53.882 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.142+0000 7f8c8d2f1640 1 -- 172.21.15.167:0/3726515570 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c80030640 con 0x7f8c88105ba0 2024-06-18T12:57:53.882 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.143+0000 7f8c90557640 1 -- 172.21.15.167:0/3726515570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c88105ba0 msgr2=0x7f8c88107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:53.882 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.143+0000 7f8c90557640 1 --2- 172.21.15.167:0/3726515570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c88105ba0 0x7f8c88107f90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009820 tx=0x7f8c8002a270 comp rx=0 tx=0).stop 2024-06-18T12:57:53.882 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.143+0000 7f8c90557640 1 -- 172.21.15.167:0/3726515570 shutdown_connections 2024-06-18T12:57:53.882 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.143+0000 7f8c90557640 1 --2- 172.21.15.167:0/3726515570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c88105ba0 0x7f8c88107f90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:53.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.143+0000 7f8c90557640 1 -- 172.21.15.167:0/3726515570 >> 172.21.15.167:0/3726515570 conn(0x7f8c880ff780 msgr2=0x7f8c88101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:53.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.143+0000 7f8c90557640 1 -- 172.21.15.167:0/3726515570 shutdown_connections 2024-06-18T12:57:53.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.143+0000 7f8c90557640 1 -- 172.21.15.167:0/3726515570 wait complete. 2024-06-18T12:57:53.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.143+0000 7f8c90557640 1 Processor -- start 2024-06-18T12:57:53.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.144+0000 7f8c90557640 1 -- start start 2024-06-18T12:57:53.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.144+0000 7f8c90557640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c88105ba0 0x7f8c88105420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:53.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.144+0000 7f8c90557640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c881093f0 con 0x7f8c88105ba0 2024-06-18T12:57:53.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.144+0000 7f8c8e2f3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c88105ba0 0x7f8c88105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:53.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.144+0000 7f8c8e2f3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c88105ba0 0x7f8c88105420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60266/0 (socket says 172.21.15.167:60266) 2024-06-18T12:57:53.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.144+0000 7f8c8e2f3640 1 -- 172.21.15.167:0/882243305 learned_addr learned my addr 172.21.15.167:0/882243305 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:53.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.144+0000 7f8c8e2f3640 1 -- 172.21.15.167:0/882243305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c88105960 con 0x7f8c88105ba0 2024-06-18T12:57:53.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.144+0000 7f8c8e2f3640 1 --2- 172.21.15.167:0/882243305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c88105ba0 0x7f8c88105420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8c80000c00 tx=0x7f8c800327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:53.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.144+0000 7f8c7f7fe640 1 -- 172.21.15.167:0/882243305 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c80037040 con 0x7f8c88105ba0 2024-06-18T12:57:53.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.144+0000 7f8c7f7fe640 1 -- 172.21.15.167:0/882243305 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f8c80040070 con 0x7f8c88105ba0 2024-06-18T12:57:53.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.144+0000 7f8c7f7fe640 1 -- 172.21.15.167:0/882243305 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c8003bda0 con 0x7f8c88105ba0 2024-06-18T12:57:53.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.144+0000 7f8c90557640 1 -- 172.21.15.167:0/882243305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c88103b50 con 0x7f8c88105ba0 2024-06-18T12:57:53.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.144+0000 7f8c90557640 1 -- 172.21.15.167:0/882243305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c88104030 con 0x7f8c88105ba0 2024-06-18T12:57:53.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.145+0000 7f8c7f7fe640 1 -- 172.21.15.167:0/882243305 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f8c80030b40 con 0x7f8c88105ba0 2024-06-18T12:57:53.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.145+0000 7f8c90557640 1 -- 172.21.15.167:0/882243305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c88106000 con 0x7f8c88105ba0 2024-06-18T12:57:53.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.145+0000 7f8c7f7fe640 1 --2- 172.21.15.167:0/882243305 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f8c74038eb0 0x7f8c7403b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:53.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.145+0000 7f8c7f7fe640 1 -- 172.21.15.167:0/882243305 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8c8006bff0 con 0x7f8c88105ba0 2024-06-18T12:57:53.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.146+0000 7f8c8daf2640 1 --2- 172.21.15.167:0/882243305 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f8c74038eb0 0x7f8c7403b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:53.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.146+0000 7f8c8daf2640 1 --2- 172.21.15.167:0/882243305 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f8c74038eb0 0x7f8c7403b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8c780098c0 tx=0x7f8c78006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:53.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.150+0000 7f8c7f7fe640 1 -- 172.21.15.167:0/882243305 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f8c80030e30 con 0x7f8c88105ba0 2024-06-18T12:57:53.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.188+0000 7f8c7f7fe640 1 -- 172.21.15.167:0/882243305 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8c8003b6f0 con 0x7f8c88105ba0 2024-06-18T12:57:53.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.344+0000 7f8c90557640 1 -- 172.21.15.167:0/882243305 --> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f8c881046c0 con 0x7f8c74038eb0 2024-06-18T12:57:53.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.352+0000 7f8c7f7fe640 1 -- 172.21.15.167:0/882243305 <== mgr.14118 v2:172.21.15.167:6800/1873453520 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8c881046c0 con 0x7f8c74038eb0 2024-06-18T12:57:53.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.354+0000 7f8c90557640 1 -- 172.21.15.167:0/882243305 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f8c74038eb0 msgr2=0x7f8c7403b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:53.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.354+0000 7f8c90557640 1 --2- 172.21.15.167:0/882243305 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f8c74038eb0 0x7f8c7403b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8c780098c0 tx=0x7f8c78006db0 comp rx=0 tx=0).stop 2024-06-18T12:57:53.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.354+0000 7f8c90557640 1 -- 172.21.15.167:0/882243305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c88105ba0 msgr2=0x7f8c88105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:53.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.354+0000 7f8c90557640 1 --2- 172.21.15.167:0/882243305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c88105ba0 0x7f8c88105420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8c80000c00 tx=0x7f8c800327b0 comp rx=0 tx=0).stop 2024-06-18T12:57:53.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.354+0000 7f8c90557640 1 -- 172.21.15.167:0/882243305 shutdown_connections 2024-06-18T12:57:53.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.354+0000 7f8c90557640 1 --2- 172.21.15.167:0/882243305 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f8c74038eb0 0x7f8c7403b370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:53.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.354+0000 7f8c90557640 1 --2- 172.21.15.167:0/882243305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c88105ba0 0x7f8c88105420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:53.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.354+0000 7f8c90557640 1 -- 172.21.15.167:0/882243305 >> 172.21.15.167:0/882243305 conn(0x7f8c880ff780 msgr2=0x7f8c88100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:53.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.355+0000 7f8c90557640 1 -- 172.21.15.167:0/882243305 shutdown_connections 2024-06-18T12:57:53.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:53.355+0000 7f8c90557640 1 -- 172.21.15.167:0/882243305 wait complete. 2024-06-18T12:57:54.254 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:54 smithi167 ceph-mon[29496]: [18/Jun/2024:12:57:53] ENGINE Bus STARTING 2024-06-18T12:57:54.255 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:54 smithi167 ceph-mon[29496]: mgrmap e8: y(active, since 2s) 2024-06-18T12:57:54.255 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:54 smithi167 ceph-mon[29496]: [18/Jun/2024:12:57:53] ENGINE Serving on https://172.21.15.167:7150 2024-06-18T12:57:54.255 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:54 smithi167 ceph-mon[29496]: [18/Jun/2024:12:57:53] ENGINE Error in HTTPServer.serve 2024-06-18T12:57:54.255 INFO:journalctl@ceph.mon.a.smithi167.stdout: Traceback (most recent call last): 2024-06-18T12:57:54.255 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-18T12:57:54.255 INFO:journalctl@ceph.mon.a.smithi167.stdout: self._connections.run(self.expiration_interval) 2024-06-18T12:57:54.255 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-18T12:57:54.255 INFO:journalctl@ceph.mon.a.smithi167.stdout: self._run(expiration_interval) 2024-06-18T12:57:54.255 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-18T12:57:54.255 INFO:journalctl@ceph.mon.a.smithi167.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-18T12:57:54.256 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-18T12:57:54.256 INFO:journalctl@ceph.mon.a.smithi167.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-18T12:57:54.256 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-18T12:57:54.256 INFO:journalctl@ceph.mon.a.smithi167.stdout: s = self.context.wrap_socket( 2024-06-18T12:57:54.256 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-18T12:57:54.256 INFO:journalctl@ceph.mon.a.smithi167.stdout: return self.sslsocket_class._create( 2024-06-18T12:57:54.256 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-18T12:57:54.256 INFO:journalctl@ceph.mon.a.smithi167.stdout: self.do_handshake() 2024-06-18T12:57:54.256 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-18T12:57:54.256 INFO:journalctl@ceph.mon.a.smithi167.stdout: self._sslobj.do_handshake() 2024-06-18T12:57:54.256 INFO:journalctl@ceph.mon.a.smithi167.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-18T12:57:54.256 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:54 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:57:54.256 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:54 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' 2024-06-18T12:57:54.256 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:54 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:57:55.012 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-18T12:57:55.012 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.255+0000 7f1307038640 1 Processor -- start 2024-06-18T12:57:55.012 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.255+0000 7f1307038640 1 -- start start 2024-06-18T12:57:55.012 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.255+0000 7f1307038640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1300105b90 0x7f1300107f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:55.012 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.255+0000 7f1307038640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1300079660 con 0x7f1300105b90 2024-06-18T12:57:55.012 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.255+0000 7f1304dd4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1300105b90 0x7f1300107f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:55.013 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.255+0000 7f1304dd4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1300105b90 0x7f1300107f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60270/0 (socket says 172.21.15.167:60270) 2024-06-18T12:57:55.013 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.255+0000 7f1304dd4640 1 -- 172.21.15.167:0/2567581875 learned_addr learned my addr 172.21.15.167:0/2567581875 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:55.013 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.255+0000 7f1304dd4640 1 -- 172.21.15.167:0/2567581875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13001084c0 con 0x7f1300105b90 2024-06-18T12:57:55.013 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.256+0000 7f1304dd4640 1 --2- 172.21.15.167:0/2567581875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1300105b90 0x7f1300107f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009820 tx=0x7f12f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69d14e12b1b3cc9 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:55.013 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.256+0000 7f12ff7fe640 1 -- 172.21.15.167:0/2567581875 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12f4037070 con 0x7f1300105b90 2024-06-18T12:57:55.013 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.256+0000 7f12ff7fe640 1 -- 172.21.15.167:0/2567581875 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f12f40328a0 con 0x7f1300105b90 2024-06-18T12:57:55.013 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.256+0000 7f12ff7fe640 1 -- 172.21.15.167:0/2567581875 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12f4030640 con 0x7f1300105b90 2024-06-18T12:57:55.013 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.257+0000 7f1307038640 1 -- 172.21.15.167:0/2567581875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1300105b90 msgr2=0x7f1300107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:55.013 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.257+0000 7f1307038640 1 --2- 172.21.15.167:0/2567581875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1300105b90 0x7f1300107f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009820 tx=0x7f12f402a270 comp rx=0 tx=0).stop 2024-06-18T12:57:55.013 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.257+0000 7f1307038640 1 -- 172.21.15.167:0/2567581875 shutdown_connections 2024-06-18T12:57:55.013 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.257+0000 7f1307038640 1 --2- 172.21.15.167:0/2567581875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1300105b90 0x7f1300107f80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:55.013 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.257+0000 7f1307038640 1 -- 172.21.15.167:0/2567581875 >> 172.21.15.167:0/2567581875 conn(0x7f13000ff7d0 msgr2=0x7f1300101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:55.013 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.257+0000 7f1307038640 1 -- 172.21.15.167:0/2567581875 shutdown_connections 2024-06-18T12:57:55.014 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.257+0000 7f1307038640 1 -- 172.21.15.167:0/2567581875 wait complete. 2024-06-18T12:57:55.014 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.257+0000 7f1307038640 1 Processor -- start 2024-06-18T12:57:55.014 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.257+0000 7f1307038640 1 -- start start 2024-06-18T12:57:55.014 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.258+0000 7f1307038640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1300105b90 0x7f1300198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:55.014 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.258+0000 7f1307038640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f13001093e0 con 0x7f1300105b90 2024-06-18T12:57:55.014 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.258+0000 7f1304dd4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1300105b90 0x7f1300198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:55.014 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.258+0000 7f1304dd4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1300105b90 0x7f1300198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60280/0 (socket says 172.21.15.167:60280) 2024-06-18T12:57:55.014 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.258+0000 7f1304dd4640 1 -- 172.21.15.167:0/2932280066 learned_addr learned my addr 172.21.15.167:0/2932280066 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:55.014 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.258+0000 7f1304dd4640 1 -- 172.21.15.167:0/2932280066 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1300199480 con 0x7f1300105b90 2024-06-18T12:57:55.014 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.258+0000 7f1304dd4640 1 --2- 172.21.15.167:0/2932280066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1300105b90 0x7f1300198f40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f12f4030380 tx=0x7f12f40326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:55.014 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.258+0000 7f12fdffb640 1 -- 172.21.15.167:0/2932280066 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12f4037040 con 0x7f1300105b90 2024-06-18T12:57:55.014 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.258+0000 7f12fdffb640 1 -- 172.21.15.167:0/2932280066 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f12f4040070 con 0x7f1300105b90 2024-06-18T12:57:55.014 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.258+0000 7f1307038640 1 -- 172.21.15.167:0/2932280066 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1300199710 con 0x7f1300105b90 2024-06-18T12:57:55.015 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.258+0000 7f12fdffb640 1 -- 172.21.15.167:0/2932280066 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12f403bca0 con 0x7f1300105b90 2024-06-18T12:57:55.015 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.258+0000 7f1307038640 1 -- 172.21.15.167:0/2932280066 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f130019c400 con 0x7f1300105b90 2024-06-18T12:57:55.015 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.259+0000 7f12fdffb640 1 -- 172.21.15.167:0/2932280066 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f12f4045460 con 0x7f1300105b90 2024-06-18T12:57:55.015 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.259+0000 7f1307038640 1 -- 172.21.15.167:0/2932280066 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1300105ff0 con 0x7f1300105b90 2024-06-18T12:57:55.015 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.260+0000 7f12fdffb640 1 --2- 172.21.15.167:0/2932280066 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f12f0038fc0 0x7f12f003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:55.015 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.260+0000 7f12fdffb640 1 -- 172.21.15.167:0/2932280066 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f12f406c1f0 con 0x7f1300105b90 2024-06-18T12:57:55.015 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.260+0000 7f12fffff640 1 --2- 172.21.15.167:0/2932280066 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f12f0038fc0 0x7f12f003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:55.015 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.260+0000 7f12fffff640 1 --2- 172.21.15.167:0/2932280066 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f12f0038fc0 0x7f12f003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0098c0 tx=0x7f12ec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:55.015 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.264+0000 7f12fdffb640 1 -- 172.21.15.167:0/2932280066 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f12f402e200 con 0x7f1300105b90 2024-06-18T12:57:55.015 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.440+0000 7f1307038640 1 -- 172.21.15.167:0/2932280066 --> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f1300100a50 con 0x7f12f0038fc0 2024-06-18T12:57:55.015 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.441+0000 7f12fdffb640 1 -- 172.21.15.167:0/2932280066 <== mgr.14118 v2:172.21.15.167:6800/1873453520 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f1300100a50 con 0x7f12f0038fc0 2024-06-18T12:57:55.015 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.444+0000 7f1307038640 1 -- 172.21.15.167:0/2932280066 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f12f0038fc0 msgr2=0x7f12f003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:55.016 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.444+0000 7f1307038640 1 --2- 172.21.15.167:0/2932280066 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f12f0038fc0 0x7f12f003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0098c0 tx=0x7f12ec006db0 comp rx=0 tx=0).stop 2024-06-18T12:57:55.016 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.444+0000 7f1307038640 1 -- 172.21.15.167:0/2932280066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1300105b90 msgr2=0x7f1300198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:55.016 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.444+0000 7f1307038640 1 --2- 172.21.15.167:0/2932280066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1300105b90 0x7f1300198f40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f12f4030380 tx=0x7f12f40326e0 comp rx=0 tx=0).stop 2024-06-18T12:57:55.016 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.444+0000 7f1307038640 1 -- 172.21.15.167:0/2932280066 shutdown_connections 2024-06-18T12:57:55.016 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.444+0000 7f1307038640 1 --2- 172.21.15.167:0/2932280066 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f12f0038fc0 0x7f12f003b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:55.016 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.444+0000 7f1307038640 1 --2- 172.21.15.167:0/2932280066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1300105b90 0x7f1300198f40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:55.016 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.444+0000 7f1307038640 1 -- 172.21.15.167:0/2932280066 >> 172.21.15.167:0/2932280066 conn(0x7f13000ff7d0 msgr2=0x7f1300100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:55.016 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.444+0000 7f1307038640 1 -- 172.21.15.167:0/2932280066 shutdown_connections 2024-06-18T12:57:55.016 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:54.444+0000 7f1307038640 1 -- 172.21.15.167:0/2932280066 wait complete. 2024-06-18T12:57:55.016 INFO:teuthology.orchestra.run.smithi167.stdout:Generating ssh key... 2024-06-18T12:57:55.332 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-mon[29496]: [18/Jun/2024:12:57:53] ENGINE Serving on http://172.21.15.167:8765 2024-06-18T12:57:55.332 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-mon[29496]: [18/Jun/2024:12:57:53] ENGINE Bus STARTED 2024-06-18T12:57:55.332 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-mon[29496]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:57:55.976 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: Generating public/private rsa key pair. 2024-06-18T12:57:55.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: Your identification has been saved in /tmp/tmpebf8iu2r/key 2024-06-18T12:57:55.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: Your public key has been saved in /tmp/tmpebf8iu2r/key.pub 2024-06-18T12:57:55.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: The key fingerprint is: 2024-06-18T12:57:55.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: SHA256:hyV5/qtBROPVp+n72lNoTkpYWHdKO0OdC6g+e7B5kI0 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:57:55.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: The key's randomart image is: 2024-06-18T12:57:55.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: +---[RSA 3072]----+ 2024-06-18T12:57:55.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: | o o. ..| 2024-06-18T12:57:55.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: | + +..=.+| 2024-06-18T12:57:55.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: | o *o +.B.| 2024-06-18T12:57:55.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: | O. . B. | 2024-06-18T12:57:55.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: | S Bo . + | 2024-06-18T12:57:55.978 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: | E.o. = .| 2024-06-18T12:57:55.978 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: | O..= ..| 2024-06-18T12:57:55.978 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: | + +..oo | 2024-06-18T12:57:55.978 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: | +.. .o+| 2024-06-18T12:57:55.978 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:57:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: +----[SHA256]-----+ 2024-06-18T12:57:56.428 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:56 smithi167 ceph-mon[29496]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:57:56.428 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:56 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' 2024-06-18T12:57:56.428 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:56 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' 2024-06-18T12:57:56.434 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.410+0000 7f90353b3640 1 Processor -- start 2024-06-18T12:57:56.434 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.410+0000 7f90353b3640 1 -- start start 2024-06-18T12:57:56.435 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.410+0000 7f90353b3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9030105c70 0x7f9030108060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:56.435 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.410+0000 7f90353b3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9030078d70 con 0x7f9030105c70 2024-06-18T12:57:56.435 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.411+0000 7f902effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9030105c70 0x7f9030108060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:56.435 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.411+0000 7f902effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9030105c70 0x7f9030108060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60294/0 (socket says 172.21.15.167:60294) 2024-06-18T12:57:56.435 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.411+0000 7f902effd640 1 -- 172.21.15.167:0/3471651438 learned_addr learned my addr 172.21.15.167:0/3471651438 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:56.435 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.411+0000 7f902effd640 1 -- 172.21.15.167:0/3471651438 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90301085a0 con 0x7f9030105c70 2024-06-18T12:57:56.435 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.412+0000 7f902effd640 1 --2- 172.21.15.167:0/3471651438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9030105c70 0x7f9030108060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9028009820 tx=0x7f902802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf9a1b193eae9c server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:56.435 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.412+0000 7f902dffb640 1 -- 172.21.15.167:0/3471651438 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9028037070 con 0x7f9030105c70 2024-06-18T12:57:56.435 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.412+0000 7f902dffb640 1 -- 172.21.15.167:0/3471651438 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f90280328a0 con 0x7f9030105c70 2024-06-18T12:57:56.435 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.412+0000 7f902dffb640 1 -- 172.21.15.167:0/3471651438 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9028030640 con 0x7f9030105c70 2024-06-18T12:57:56.436 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.412+0000 7f90353b3640 1 -- 172.21.15.167:0/3471651438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9030105c70 msgr2=0x7f9030108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:56.436 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.412+0000 7f90353b3640 1 --2- 172.21.15.167:0/3471651438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9030105c70 0x7f9030108060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9028009820 tx=0x7f902802a270 comp rx=0 tx=0).stop 2024-06-18T12:57:56.436 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.412+0000 7f90353b3640 1 -- 172.21.15.167:0/3471651438 shutdown_connections 2024-06-18T12:57:56.436 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.412+0000 7f90353b3640 1 --2- 172.21.15.167:0/3471651438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9030105c70 0x7f9030108060 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:56.436 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.412+0000 7f90353b3640 1 -- 172.21.15.167:0/3471651438 >> 172.21.15.167:0/3471651438 conn(0x7f90300ff790 msgr2=0x7f9030101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:56.436 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.412+0000 7f90353b3640 1 -- 172.21.15.167:0/3471651438 shutdown_connections 2024-06-18T12:57:56.436 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.412+0000 7f90353b3640 1 -- 172.21.15.167:0/3471651438 wait complete. 2024-06-18T12:57:56.436 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.413+0000 7f90353b3640 1 Processor -- start 2024-06-18T12:57:56.436 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.413+0000 7f90353b3640 1 -- start start 2024-06-18T12:57:56.436 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.413+0000 7f90353b3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9030105c70 0x7f9030199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:56.436 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.413+0000 7f90353b3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9030109460 con 0x7f9030105c70 2024-06-18T12:57:56.436 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.413+0000 7f902effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9030105c70 0x7f9030199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:56.436 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.414+0000 7f902effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9030105c70 0x7f9030199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60298/0 (socket says 172.21.15.167:60298) 2024-06-18T12:57:56.436 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.414+0000 7f902effd640 1 -- 172.21.15.167:0/2296955314 learned_addr learned my addr 172.21.15.167:0/2296955314 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:56.437 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.414+0000 7f902effd640 1 -- 172.21.15.167:0/2296955314 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90301995c0 con 0x7f9030105c70 2024-06-18T12:57:56.437 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.414+0000 7f902effd640 1 --2- 172.21.15.167:0/2296955314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9030105c70 0x7f9030199080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9028000c00 tx=0x7f90280326c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:56.437 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.414+0000 7f9027fff640 1 -- 172.21.15.167:0/2296955314 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9028037040 con 0x7f9030105c70 2024-06-18T12:57:56.437 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.414+0000 7f9027fff640 1 -- 172.21.15.167:0/2296955314 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f9028040070 con 0x7f9030105c70 2024-06-18T12:57:56.437 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.414+0000 7f90353b3640 1 -- 172.21.15.167:0/2296955314 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9030199850 con 0x7f9030105c70 2024-06-18T12:57:56.437 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.414+0000 7f9027fff640 1 -- 172.21.15.167:0/2296955314 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f902803bd40 con 0x7f9030105c70 2024-06-18T12:57:56.437 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.414+0000 7f90353b3640 1 -- 172.21.15.167:0/2296955314 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903019c540 con 0x7f9030105c70 2024-06-18T12:57:56.437 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.415+0000 7f9027fff640 1 -- 172.21.15.167:0/2296955314 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9028045460 con 0x7f9030105c70 2024-06-18T12:57:56.437 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.415+0000 7f90353b3640 1 -- 172.21.15.167:0/2296955314 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90301060d0 con 0x7f9030105c70 2024-06-18T12:57:56.437 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.415+0000 7f9027fff640 1 --2- 172.21.15.167:0/2296955314 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f901c038f70 0x7f901c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:56.437 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.415+0000 7f9027fff640 1 -- 172.21.15.167:0/2296955314 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f902806c310 con 0x7f9030105c70 2024-06-18T12:57:56.437 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.415+0000 7f902e7fc640 1 --2- 172.21.15.167:0/2296955314 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f901c038f70 0x7f901c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:56.437 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.416+0000 7f902e7fc640 1 --2- 172.21.15.167:0/2296955314 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f901c038f70 0x7f901c03b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f90180098c0 tx=0x7f9018006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:56.438 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.420+0000 7f9027fff640 1 -- 172.21.15.167:0/2296955314 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f9028045850 con 0x7f9030105c70 2024-06-18T12:57:56.438 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.596+0000 7f90353b3640 1 -- 172.21.15.167:0/2296955314 --> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f9030100b80 con 0x7f901c038f70 2024-06-18T12:57:56.438 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.829+0000 7f9027fff640 1 -- 172.21.15.167:0/2296955314 <== mgr.14118 v2:172.21.15.167:6800/1873453520 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9030100b80 con 0x7f901c038f70 2024-06-18T12:57:56.438 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.831+0000 7f90353b3640 1 -- 172.21.15.167:0/2296955314 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f901c038f70 msgr2=0x7f901c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:56.438 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.831+0000 7f90353b3640 1 --2- 172.21.15.167:0/2296955314 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f901c038f70 0x7f901c03b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f90180098c0 tx=0x7f9018006db0 comp rx=0 tx=0).stop 2024-06-18T12:57:56.438 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.831+0000 7f90353b3640 1 -- 172.21.15.167:0/2296955314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9030105c70 msgr2=0x7f9030199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:56.438 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.831+0000 7f90353b3640 1 --2- 172.21.15.167:0/2296955314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9030105c70 0x7f9030199080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9028000c00 tx=0x7f90280326c0 comp rx=0 tx=0).stop 2024-06-18T12:57:56.438 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.831+0000 7f90353b3640 1 -- 172.21.15.167:0/2296955314 shutdown_connections 2024-06-18T12:57:56.438 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.831+0000 7f90353b3640 1 --2- 172.21.15.167:0/2296955314 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f901c038f70 0x7f901c03b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:56.438 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.831+0000 7f90353b3640 1 --2- 172.21.15.167:0/2296955314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9030105c70 0x7f9030199080 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:56.438 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.831+0000 7f90353b3640 1 -- 172.21.15.167:0/2296955314 >> 172.21.15.167:0/2296955314 conn(0x7f90300ff790 msgr2=0x7f9030100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:56.438 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.832+0000 7f90353b3640 1 -- 172.21.15.167:0/2296955314 shutdown_connections 2024-06-18T12:57:56.439 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:55.832+0000 7f90353b3640 1 -- 172.21.15.167:0/2296955314 wait complete. 2024-06-18T12:57:57.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:57 smithi167 ceph-mon[29496]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:57:57.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:57 smithi167 ceph-mon[29496]: Generating ssh key... 2024-06-18T12:57:57.584 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/0udb4frrnW2NbdrCUUBbKqccVq5E0FpRcuDdYbtYNAVJ0xQPC8KgRmPYY1C3HVsi0YKYwtvsI0R99zRw5oaSv/XH/6zXlJpU9mCGu8av/nwUl3Dr5P2QmxSreGvsMT/RcV+hdip9BL75fxgEGgrgYBnFkrIo1eE7z5YXhajTU1vGRhp/nEe993cWJNRiGyQpTG6L+L7MUe3pPQ4EJUWkIe110ghu+3nj9sz3RWkBR1UbUGjZXCZ8kQCFGkRhvzanexgsKBuWVonCMb/+D10dl8KIKRI5ZnVQ9shzS9qM3KSZtpZexLevvclo7eIVAJhEaItpKlMZPsL0QK0IqnVO6euOCRMAxjEZdWZJttdYBlX85tTh67Ljs7m7UW9nuDSdF6SfS/7JwWXIpyg4gaseVHeVG5u0K8nXfJCNbVUOf8GEM7aKbXmWVzBqNHAd21gzAPVfZn7NoX5471D8bKnMk+AjZQIA6bDHWs3ct1JLZz/2U9T+qL5sRoBpDmWB+Vs= ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:57:57.584 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.808+0000 7f38ce8f6640 1 Processor -- start 2024-06-18T12:57:57.584 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.808+0000 7f38ce8f6640 1 -- start start 2024-06-18T12:57:57.585 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.808+0000 7f38ce8f6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38c8105110 0x7f38c8105510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:57.585 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.808+0000 7f38ce8f6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f38c8105ae0 con 0x7f38c8105110 2024-06-18T12:57:57.585 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.809+0000 7f38c7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38c8105110 0x7f38c8105510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:57.585 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.809+0000 7f38c7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38c8105110 0x7f38c8105510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60300/0 (socket says 172.21.15.167:60300) 2024-06-18T12:57:57.585 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.809+0000 7f38c7fff640 1 -- 172.21.15.167:0/3627512966 learned_addr learned my addr 172.21.15.167:0/3627512966 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:57.585 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.809+0000 7f38c7fff640 1 -- 172.21.15.167:0/3627512966 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c81062c0 con 0x7f38c8105110 2024-06-18T12:57:57.585 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.810+0000 7f38c7fff640 1 --2- 172.21.15.167:0/3627512966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38c8105110 0x7f38c8105510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009820 tx=0x7f38b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffaf730ab0cfd67 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:57.585 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.810+0000 7f38c77fe640 1 -- 172.21.15.167:0/3627512966 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38b8037070 con 0x7f38c8105110 2024-06-18T12:57:57.585 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.810+0000 7f38c77fe640 1 -- 172.21.15.167:0/3627512966 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f38b80328a0 con 0x7f38c8105110 2024-06-18T12:57:57.585 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.810+0000 7f38c77fe640 1 -- 172.21.15.167:0/3627512966 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38b8030640 con 0x7f38c8105110 2024-06-18T12:57:57.585 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.810+0000 7f38ce8f6640 1 -- 172.21.15.167:0/3627512966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38c8105110 msgr2=0x7f38c8105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:57.585 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.810+0000 7f38ce8f6640 1 --2- 172.21.15.167:0/3627512966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38c8105110 0x7f38c8105510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009820 tx=0x7f38b802a270 comp rx=0 tx=0).stop 2024-06-18T12:57:57.585 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.810+0000 7f38ce8f6640 1 -- 172.21.15.167:0/3627512966 shutdown_connections 2024-06-18T12:57:57.586 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.810+0000 7f38ce8f6640 1 --2- 172.21.15.167:0/3627512966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38c8105110 0x7f38c8105510 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:57.586 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.810+0000 7f38ce8f6640 1 -- 172.21.15.167:0/3627512966 >> 172.21.15.167:0/3627512966 conn(0x7f38c81008a0 msgr2=0x7f38c8102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:57.586 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.811+0000 7f38ce8f6640 1 -- 172.21.15.167:0/3627512966 shutdown_connections 2024-06-18T12:57:57.586 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.811+0000 7f38ce8f6640 1 -- 172.21.15.167:0/3627512966 wait complete. 2024-06-18T12:57:57.586 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.811+0000 7f38ce8f6640 1 Processor -- start 2024-06-18T12:57:57.586 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.811+0000 7f38ce8f6640 1 -- start start 2024-06-18T12:57:57.586 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.811+0000 7f38ce8f6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38c8105110 0x7f38c8198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:57.586 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.811+0000 7f38ce8f6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f38c81094f0 con 0x7f38c8105110 2024-06-18T12:57:57.586 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.811+0000 7f38c7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38c8105110 0x7f38c8198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:57.586 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.811+0000 7f38c7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38c8105110 0x7f38c8198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60306/0 (socket says 172.21.15.167:60306) 2024-06-18T12:57:57.586 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.811+0000 7f38c7fff640 1 -- 172.21.15.167:0/4144385687 learned_addr learned my addr 172.21.15.167:0/4144385687 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:57:57.586 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.812+0000 7f38c7fff640 1 -- 172.21.15.167:0/4144385687 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c81994b0 con 0x7f38c8105110 2024-06-18T12:57:57.587 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.812+0000 7f38c7fff640 1 --2- 172.21.15.167:0/4144385687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38c8105110 0x7f38c8198f70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f38b8030380 tx=0x7f38b80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:57.587 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.812+0000 7f38c5ffb640 1 -- 172.21.15.167:0/4144385687 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38b8037040 con 0x7f38c8105110 2024-06-18T12:57:57.587 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.812+0000 7f38ce8f6640 1 -- 172.21.15.167:0/4144385687 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c8199740 con 0x7f38c8105110 2024-06-18T12:57:57.587 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.812+0000 7f38c5ffb640 1 -- 172.21.15.167:0/4144385687 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f38b8040070 con 0x7f38c8105110 2024-06-18T12:57:57.587 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.812+0000 7f38c5ffb640 1 -- 172.21.15.167:0/4144385687 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38b803bc70 con 0x7f38c8105110 2024-06-18T12:57:57.587 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.812+0000 7f38ce8f6640 1 -- 172.21.15.167:0/4144385687 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c819c430 con 0x7f38c8105110 2024-06-18T12:57:57.587 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.813+0000 7f38c5ffb640 1 -- 172.21.15.167:0/4144385687 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f38b8045460 con 0x7f38c8105110 2024-06-18T12:57:57.587 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.813+0000 7f38ce8f6640 1 -- 172.21.15.167:0/4144385687 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c8105590 con 0x7f38c8105110 2024-06-18T12:57:57.587 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.813+0000 7f38c5ffb640 1 --2- 172.21.15.167:0/4144385687 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f38b4038f70 0x7f38b403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:57:57.587 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.813+0000 7f38c5ffb640 1 -- 172.21.15.167:0/4144385687 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f38b806c3a0 con 0x7f38c8105110 2024-06-18T12:57:57.587 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.813+0000 7f38bffff640 1 --2- 172.21.15.167:0/4144385687 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f38b4038f70 0x7f38b403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:57:57.587 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.814+0000 7f38bffff640 1 --2- 172.21.15.167:0/4144385687 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f38b4038f70 0x7f38b403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f38b00098c0 tx=0x7f38b0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:57:57.587 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.818+0000 7f38c5ffb640 1 -- 172.21.15.167:0/4144385687 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f38b8043520 con 0x7f38c8105110 2024-06-18T12:57:57.587 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.997+0000 7f38ce8f6640 1 -- 172.21.15.167:0/4144385687 --> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f38c8101b50 con 0x7f38b4038f70 2024-06-18T12:57:57.588 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:56.997+0000 7f38c5ffb640 1 -- 172.21.15.167:0/4144385687 <== mgr.14118 v2:172.21.15.167:6800/1873453520 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f38c8101b50 con 0x7f38b4038f70 2024-06-18T12:57:57.588 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.000+0000 7f38ce8f6640 1 -- 172.21.15.167:0/4144385687 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f38b4038f70 msgr2=0x7f38b403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:57.588 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.000+0000 7f38ce8f6640 1 --2- 172.21.15.167:0/4144385687 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f38b4038f70 0x7f38b403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f38b00098c0 tx=0x7f38b0006db0 comp rx=0 tx=0).stop 2024-06-18T12:57:57.588 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.000+0000 7f38ce8f6640 1 -- 172.21.15.167:0/4144385687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38c8105110 msgr2=0x7f38c8198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:57:57.588 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.000+0000 7f38ce8f6640 1 --2- 172.21.15.167:0/4144385687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38c8105110 0x7f38c8198f70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f38b8030380 tx=0x7f38b80326e0 comp rx=0 tx=0).stop 2024-06-18T12:57:57.588 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.001+0000 7f38ce8f6640 1 -- 172.21.15.167:0/4144385687 shutdown_connections 2024-06-18T12:57:57.588 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.001+0000 7f38ce8f6640 1 --2- 172.21.15.167:0/4144385687 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f38b4038f70 0x7f38b403b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:57.588 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.001+0000 7f38ce8f6640 1 --2- 172.21.15.167:0/4144385687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38c8105110 0x7f38c8198f70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:57:57.588 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.001+0000 7f38ce8f6640 1 -- 172.21.15.167:0/4144385687 >> 172.21.15.167:0/4144385687 conn(0x7f38c81008a0 msgr2=0x7f38c8101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:57:57.588 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.001+0000 7f38ce8f6640 1 -- 172.21.15.167:0/4144385687 shutdown_connections 2024-06-18T12:57:57.588 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.001+0000 7f38ce8f6640 1 -- 172.21.15.167:0/4144385687 wait complete. 2024-06-18T12:57:57.588 INFO:teuthology.orchestra.run.smithi167.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-18T12:57:57.588 INFO:teuthology.orchestra.run.smithi167.stdout:Adding key to root@localhost authorized_keys... 2024-06-18T12:57:57.589 INFO:teuthology.orchestra.run.smithi167.stdout:Adding host smithi167... 2024-06-18T12:57:58.397 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:58 smithi167 ceph-mon[29496]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:57:59.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:57:59 smithi167 ceph-mon[29496]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "addr": "172.21.15.167", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:58:00.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:00 smithi167 ceph-mon[29496]: Deploying cephadm binary to smithi167 2024-06-18T12:58:01.833 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout Added host 'smithi167' with addr '172.21.15.167' 2024-06-18T12:58:01.833 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.954+0000 7f69f7d85640 1 Processor -- start 2024-06-18T12:58:01.833 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.954+0000 7f69f7d85640 1 -- start start 2024-06-18T12:58:01.834 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.954+0000 7f69f7d85640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69f0105070 0x7f69f0105470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:01.834 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.954+0000 7f69f7d85640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0105a40 con 0x7f69f0105070 2024-06-18T12:58:01.834 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.954+0000 7f69f5b21640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69f0105070 0x7f69f0105470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:01.834 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.954+0000 7f69f5b21640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69f0105070 0x7f69f0105470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60312/0 (socket says 172.21.15.167:60312) 2024-06-18T12:58:01.834 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.954+0000 7f69f5b21640 1 -- 172.21.15.167:0/2134652924 learned_addr learned my addr 172.21.15.167:0/2134652924 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:01.834 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.954+0000 7f69f5b21640 1 -- 172.21.15.167:0/2134652924 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f0106270 con 0x7f69f0105070 2024-06-18T12:58:01.834 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.955+0000 7f69f5b21640 1 --2- 172.21.15.167:0/2134652924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69f0105070 0x7f69f0105470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69ec009820 tx=0x7f69ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856c2b98eb9639a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:01.834 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.955+0000 7f69f4b1f640 1 -- 172.21.15.167:0/2134652924 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ec037070 con 0x7f69f0105070 2024-06-18T12:58:01.834 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.955+0000 7f69f4b1f640 1 -- 172.21.15.167:0/2134652924 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f69ec0328a0 con 0x7f69f0105070 2024-06-18T12:58:01.834 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.955+0000 7f69f4b1f640 1 -- 172.21.15.167:0/2134652924 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ec030640 con 0x7f69f0105070 2024-06-18T12:58:01.834 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.956+0000 7f69f7d85640 1 -- 172.21.15.167:0/2134652924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69f0105070 msgr2=0x7f69f0105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:01.834 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.956+0000 7f69f7d85640 1 --2- 172.21.15.167:0/2134652924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69f0105070 0x7f69f0105470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69ec009820 tx=0x7f69ec02a270 comp rx=0 tx=0).stop 2024-06-18T12:58:01.834 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.956+0000 7f69f7d85640 1 -- 172.21.15.167:0/2134652924 shutdown_connections 2024-06-18T12:58:01.835 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.956+0000 7f69f7d85640 1 --2- 172.21.15.167:0/2134652924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69f0105070 0x7f69f0105470 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:01.835 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.956+0000 7f69f7d85640 1 -- 172.21.15.167:0/2134652924 >> 172.21.15.167:0/2134652924 conn(0x7f69f01007e0 msgr2=0x7f69f0102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:01.835 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.956+0000 7f69f7d85640 1 -- 172.21.15.167:0/2134652924 shutdown_connections 2024-06-18T12:58:01.835 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.956+0000 7f69f7d85640 1 -- 172.21.15.167:0/2134652924 wait complete. 2024-06-18T12:58:01.835 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.957+0000 7f69f7d85640 1 Processor -- start 2024-06-18T12:58:01.835 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.957+0000 7f69f7d85640 1 -- start start 2024-06-18T12:58:01.835 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.957+0000 7f69f7d85640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69f0105070 0x7f69f0198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:01.835 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.957+0000 7f69f7d85640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69f01094a0 con 0x7f69f0105070 2024-06-18T12:58:01.835 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.957+0000 7f69f5b21640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69f0105070 0x7f69f0198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:01.835 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.957+0000 7f69f5b21640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69f0105070 0x7f69f0198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60320/0 (socket says 172.21.15.167:60320) 2024-06-18T12:58:01.835 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.957+0000 7f69f5b21640 1 -- 172.21.15.167:0/3940880035 learned_addr learned my addr 172.21.15.167:0/3940880035 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:01.835 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.957+0000 7f69f5b21640 1 -- 172.21.15.167:0/3940880035 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f01994e0 con 0x7f69f0105070 2024-06-18T12:58:01.835 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.957+0000 7f69f5b21640 1 --2- 172.21.15.167:0/3940880035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69f0105070 0x7f69f0198fa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69ec030380 tx=0x7f69ec0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:01.836 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.957+0000 7f69e6ffd640 1 -- 172.21.15.167:0/3940880035 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ec037040 con 0x7f69f0105070 2024-06-18T12:58:01.836 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.958+0000 7f69e6ffd640 1 -- 172.21.15.167:0/3940880035 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f69ec040070 con 0x7f69f0105070 2024-06-18T12:58:01.836 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.958+0000 7f69f7d85640 1 -- 172.21.15.167:0/3940880035 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f0199770 con 0x7f69f0105070 2024-06-18T12:58:01.836 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.958+0000 7f69e6ffd640 1 -- 172.21.15.167:0/3940880035 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ec03bc70 con 0x7f69f0105070 2024-06-18T12:58:01.836 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.958+0000 7f69f7d85640 1 -- 172.21.15.167:0/3940880035 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f019c460 con 0x7f69f0105070 2024-06-18T12:58:01.836 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.958+0000 7f69e6ffd640 1 -- 172.21.15.167:0/3940880035 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f69ec045460 con 0x7f69f0105070 2024-06-18T12:58:01.836 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.959+0000 7f69f7d85640 1 -- 172.21.15.167:0/3940880035 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f01054f0 con 0x7f69f0105070 2024-06-18T12:58:01.836 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.959+0000 7f69e6ffd640 1 --2- 172.21.15.167:0/3940880035 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f69dc038fc0 0x7f69dc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:01.836 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.959+0000 7f69e6ffd640 1 -- 172.21.15.167:0/3940880035 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f69ec06c1c0 con 0x7f69f0105070 2024-06-18T12:58:01.836 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.959+0000 7f69f5320640 1 --2- 172.21.15.167:0/3940880035 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f69dc038fc0 0x7f69dc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:01.836 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.959+0000 7f69f5320640 1 --2- 172.21.15.167:0/3940880035 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f69dc038fc0 0x7f69dc03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f69e00098c0 tx=0x7f69e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:01.836 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:57.963+0000 7f69e6ffd640 1 -- 172.21.15.167:0/3940880035 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f69ec02e200 con 0x7f69f0105070 2024-06-18T12:58:01.836 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:57:58.141+0000 7f69f7d85640 1 -- 172.21.15.167:0/3940880035 --> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi167", "addr": "172.21.15.167", "target": ["mon-mgr", ""]}) -- 0x7f69f0101b10 con 0x7f69dc038fc0 2024-06-18T12:58:01.836 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:01.349+0000 7f69e6ffd640 1 -- 172.21.15.167:0/3940880035 <== mgr.14118 v2:172.21.15.167:6800/1873453520 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f69f0101b10 con 0x7f69dc038fc0 2024-06-18T12:58:01.837 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:01.352+0000 7f69f7d85640 1 -- 172.21.15.167:0/3940880035 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f69dc038fc0 msgr2=0x7f69dc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:01.837 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:01.352+0000 7f69f7d85640 1 --2- 172.21.15.167:0/3940880035 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f69dc038fc0 0x7f69dc03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f69e00098c0 tx=0x7f69e0006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:01.837 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:01.352+0000 7f69f7d85640 1 -- 172.21.15.167:0/3940880035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69f0105070 msgr2=0x7f69f0198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:01.837 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:01.352+0000 7f69f7d85640 1 --2- 172.21.15.167:0/3940880035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69f0105070 0x7f69f0198fa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69ec030380 tx=0x7f69ec0326e0 comp rx=0 tx=0).stop 2024-06-18T12:58:01.837 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:01.352+0000 7f69f7d85640 1 -- 172.21.15.167:0/3940880035 shutdown_connections 2024-06-18T12:58:01.837 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:01.352+0000 7f69f7d85640 1 --2- 172.21.15.167:0/3940880035 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f69dc038fc0 0x7f69dc03b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:01.837 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:01.352+0000 7f69f7d85640 1 --2- 172.21.15.167:0/3940880035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69f0105070 0x7f69f0198fa0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:01.837 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:01.352+0000 7f69f7d85640 1 -- 172.21.15.167:0/3940880035 >> 172.21.15.167:0/3940880035 conn(0x7f69f01007e0 msgr2=0x7f69f0101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:01.837 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:01.352+0000 7f69f7d85640 1 -- 172.21.15.167:0/3940880035 shutdown_connections 2024-06-18T12:58:01.837 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:01.352+0000 7f69f7d85640 1 -- 172.21.15.167:0/3940880035 wait complete. 2024-06-18T12:58:01.837 INFO:teuthology.orchestra.run.smithi167.stdout:Deploying unmanaged mon service... 2024-06-18T12:58:02.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:02 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' 2024-06-18T12:58:02.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:02 smithi167 ceph-mon[29496]: Added host smithi167 2024-06-18T12:58:02.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:02 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:58:03.193 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-18T12:58:03.193 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.220+0000 7f096280e640 1 Processor -- start 2024-06-18T12:58:03.193 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.220+0000 7f096280e640 1 -- start start 2024-06-18T12:58:03.193 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.221+0000 7f096280e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f095c103fa0 0x7f095c1043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:03.193 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.221+0000 7f096280e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f095c104970 con 0x7f095c103fa0 2024-06-18T12:58:03.193 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.221+0000 7f095bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f095c103fa0 0x7f095c1043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:03.194 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.221+0000 7f095bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f095c103fa0 0x7f095c1043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60322/0 (socket says 172.21.15.167:60322) 2024-06-18T12:58:03.194 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.221+0000 7f095bfff640 1 -- 172.21.15.167:0/3381409644 learned_addr learned my addr 172.21.15.167:0/3381409644 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:03.194 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.221+0000 7f095bfff640 1 -- 172.21.15.167:0/3381409644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c1051a0 con 0x7f095c103fa0 2024-06-18T12:58:03.194 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.222+0000 7f095bfff640 1 --2- 172.21.15.167:0/3381409644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f095c103fa0 0x7f095c1043a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f094c009820 tx=0x7f094c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc44965384c40389 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:03.194 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.222+0000 7f095b7fe640 1 -- 172.21.15.167:0/3381409644 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f094c037070 con 0x7f095c103fa0 2024-06-18T12:58:03.194 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.222+0000 7f095b7fe640 1 -- 172.21.15.167:0/3381409644 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f094c0328a0 con 0x7f095c103fa0 2024-06-18T12:58:03.194 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.222+0000 7f095b7fe640 1 -- 172.21.15.167:0/3381409644 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f094c030640 con 0x7f095c103fa0 2024-06-18T12:58:03.194 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.223+0000 7f096280e640 1 -- 172.21.15.167:0/3381409644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f095c103fa0 msgr2=0x7f095c1043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:03.194 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.223+0000 7f096280e640 1 --2- 172.21.15.167:0/3381409644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f095c103fa0 0x7f095c1043a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f094c009820 tx=0x7f094c02a270 comp rx=0 tx=0).stop 2024-06-18T12:58:03.194 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.223+0000 7f096280e640 1 -- 172.21.15.167:0/3381409644 shutdown_connections 2024-06-18T12:58:03.194 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.223+0000 7f096280e640 1 --2- 172.21.15.167:0/3381409644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f095c103fa0 0x7f095c1043a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:03.194 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.223+0000 7f096280e640 1 -- 172.21.15.167:0/3381409644 >> 172.21.15.167:0/3381409644 conn(0x7f095c0ff750 msgr2=0x7f095c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:03.194 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.223+0000 7f096280e640 1 -- 172.21.15.167:0/3381409644 shutdown_connections 2024-06-18T12:58:03.195 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.223+0000 7f096280e640 1 -- 172.21.15.167:0/3381409644 wait complete. 2024-06-18T12:58:03.195 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.223+0000 7f096280e640 1 Processor -- start 2024-06-18T12:58:03.195 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.223+0000 7f096280e640 1 -- start start 2024-06-18T12:58:03.195 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.223+0000 7f096280e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f095c103fa0 0x7f095c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:03.195 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.223+0000 7f096280e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1083d0 con 0x7f095c103fa0 2024-06-18T12:58:03.195 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.224+0000 7f095bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f095c103fa0 0x7f095c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:03.195 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.224+0000 7f095bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f095c103fa0 0x7f095c198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60338/0 (socket says 172.21.15.167:60338) 2024-06-18T12:58:03.195 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.224+0000 7f095bfff640 1 -- 172.21.15.167:0/2998194664 learned_addr learned my addr 172.21.15.167:0/2998194664 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:03.195 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.224+0000 7f095bfff640 1 -- 172.21.15.167:0/2998194664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c1994e0 con 0x7f095c103fa0 2024-06-18T12:58:03.195 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.224+0000 7f095bfff640 1 --2- 172.21.15.167:0/2998194664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f095c103fa0 0x7f095c198fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f094c000c00 tx=0x7f094c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:03.195 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.224+0000 7f0959ffb640 1 -- 172.21.15.167:0/2998194664 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f094c037040 con 0x7f095c103fa0 2024-06-18T12:58:03.195 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.224+0000 7f0959ffb640 1 -- 172.21.15.167:0/2998194664 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f094c040070 con 0x7f095c103fa0 2024-06-18T12:58:03.195 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.224+0000 7f096280e640 1 -- 172.21.15.167:0/2998194664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c199770 con 0x7f095c103fa0 2024-06-18T12:58:03.196 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.224+0000 7f0959ffb640 1 -- 172.21.15.167:0/2998194664 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f094c03bda0 con 0x7f095c103fa0 2024-06-18T12:58:03.196 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.224+0000 7f096280e640 1 -- 172.21.15.167:0/2998194664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c19c460 con 0x7f095c103fa0 2024-06-18T12:58:03.196 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.225+0000 7f0959ffb640 1 -- 172.21.15.167:0/2998194664 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f094c045460 con 0x7f095c103fa0 2024-06-18T12:58:03.196 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.225+0000 7f0959ffb640 1 --2- 172.21.15.167:0/2998194664 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f0948038fc0 0x7f094803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:03.196 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.225+0000 7f096280e640 1 -- 172.21.15.167:0/2998194664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f095c104420 con 0x7f095c103fa0 2024-06-18T12:58:03.196 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.225+0000 7f0959ffb640 1 -- 172.21.15.167:0/2998194664 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f094c06c2a0 con 0x7f095c103fa0 2024-06-18T12:58:03.196 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.225+0000 7f0953fff640 1 --2- 172.21.15.167:0/2998194664 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f0948038fc0 0x7f094803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:03.196 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.226+0000 7f0953fff640 1 --2- 172.21.15.167:0/2998194664 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f0948038fc0 0x7f094803b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f09440098c0 tx=0x7f0944006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:03.196 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.230+0000 7f0959ffb640 1 -- 172.21.15.167:0/2998194664 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f094c043720 con 0x7f095c103fa0 2024-06-18T12:58:03.196 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.435+0000 7f096280e640 1 -- 172.21.15.167:0/2998194664 --> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f095c02d3f0 con 0x7f0948038fc0 2024-06-18T12:58:03.196 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.439+0000 7f0959ffb640 1 -- 172.21.15.167:0/2998194664 <== mgr.14118 v2:172.21.15.167:6800/1873453520 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f095c02d3f0 con 0x7f0948038fc0 2024-06-18T12:58:03.196 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.441+0000 7f096280e640 1 -- 172.21.15.167:0/2998194664 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f0948038fc0 msgr2=0x7f094803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:03.196 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.441+0000 7f096280e640 1 --2- 172.21.15.167:0/2998194664 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f0948038fc0 0x7f094803b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f09440098c0 tx=0x7f0944006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:03.197 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.442+0000 7f096280e640 1 -- 172.21.15.167:0/2998194664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f095c103fa0 msgr2=0x7f095c198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:03.197 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.442+0000 7f096280e640 1 --2- 172.21.15.167:0/2998194664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f095c103fa0 0x7f095c198fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f094c000c00 tx=0x7f094c0327b0 comp rx=0 tx=0).stop 2024-06-18T12:58:03.197 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.442+0000 7f096280e640 1 -- 172.21.15.167:0/2998194664 shutdown_connections 2024-06-18T12:58:03.197 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.442+0000 7f096280e640 1 --2- 172.21.15.167:0/2998194664 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f0948038fc0 0x7f094803b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:03.197 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.442+0000 7f096280e640 1 --2- 172.21.15.167:0/2998194664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f095c103fa0 0x7f095c198fa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:03.197 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.442+0000 7f096280e640 1 -- 172.21.15.167:0/2998194664 >> 172.21.15.167:0/2998194664 conn(0x7f095c0ff750 msgr2=0x7f095c100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:03.197 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.442+0000 7f096280e640 1 -- 172.21.15.167:0/2998194664 shutdown_connections 2024-06-18T12:58:03.197 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:02.442+0000 7f096280e640 1 -- 172.21.15.167:0/2998194664 wait complete. 2024-06-18T12:58:03.197 INFO:teuthology.orchestra.run.smithi167.stdout:Deploying unmanaged mgr service... 2024-06-18T12:58:03.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:03 smithi167 ceph-mon[29496]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:58:03.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:03 smithi167 ceph-mon[29496]: Saving service mon spec with placement count:5 2024-06-18T12:58:03.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:03 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' 2024-06-18T12:58:04.704 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-18T12:58:04.704 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.607+0000 7f5c3a0d4640 1 Processor -- start 2024-06-18T12:58:04.704 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.607+0000 7f5c3a0d4640 1 -- start start 2024-06-18T12:58:04.704 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.607+0000 7f5c3a0d4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c341050c0 0x7f5c341054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:04.704 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.607+0000 7f5c3a0d4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34105a90 con 0x7f5c341050c0 2024-06-18T12:58:04.704 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.608+0000 7f5c337fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c341050c0 0x7f5c341054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:04.704 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.608+0000 7f5c337fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c341050c0 0x7f5c341054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35492/0 (socket says 172.21.15.167:35492) 2024-06-18T12:58:04.705 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.608+0000 7f5c337fe640 1 -- 172.21.15.167:0/1273542576 learned_addr learned my addr 172.21.15.167:0/1273542576 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:04.705 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.608+0000 7f5c337fe640 1 -- 172.21.15.167:0/1273542576 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c341062c0 con 0x7f5c341050c0 2024-06-18T12:58:04.705 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.608+0000 7f5c337fe640 1 --2- 172.21.15.167:0/1273542576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c341050c0 0x7f5c341054c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009820 tx=0x7f5c2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eaa26822ee5b365 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:04.705 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.609+0000 7f5c32ffd640 1 -- 172.21.15.167:0/1273542576 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c24037070 con 0x7f5c341050c0 2024-06-18T12:58:04.705 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.609+0000 7f5c32ffd640 1 -- 172.21.15.167:0/1273542576 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f5c240328a0 con 0x7f5c341050c0 2024-06-18T12:58:04.705 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.609+0000 7f5c32ffd640 1 -- 172.21.15.167:0/1273542576 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c24030640 con 0x7f5c341050c0 2024-06-18T12:58:04.705 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.609+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/1273542576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c341050c0 msgr2=0x7f5c341054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:04.705 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.609+0000 7f5c3a0d4640 1 --2- 172.21.15.167:0/1273542576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c341050c0 0x7f5c341054c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009820 tx=0x7f5c2402a270 comp rx=0 tx=0).stop 2024-06-18T12:58:04.705 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.609+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/1273542576 shutdown_connections 2024-06-18T12:58:04.705 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.609+0000 7f5c3a0d4640 1 --2- 172.21.15.167:0/1273542576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c341050c0 0x7f5c341054c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:04.705 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.609+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/1273542576 >> 172.21.15.167:0/1273542576 conn(0x7f5c341008b0 msgr2=0x7f5c34102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:04.705 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.609+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/1273542576 shutdown_connections 2024-06-18T12:58:04.706 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.609+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/1273542576 wait complete. 2024-06-18T12:58:04.706 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.610+0000 7f5c3a0d4640 1 Processor -- start 2024-06-18T12:58:04.706 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.610+0000 7f5c3a0d4640 1 -- start start 2024-06-18T12:58:04.706 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.610+0000 7f5c3a0d4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c341050c0 0x7f5c34198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:04.706 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.610+0000 7f5c3a0d4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5c341094f0 con 0x7f5c341050c0 2024-06-18T12:58:04.706 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.610+0000 7f5c337fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c341050c0 0x7f5c34198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:04.706 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.610+0000 7f5c337fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c341050c0 0x7f5c34198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35494/0 (socket says 172.21.15.167:35494) 2024-06-18T12:58:04.706 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.610+0000 7f5c337fe640 1 -- 172.21.15.167:0/4104956593 learned_addr learned my addr 172.21.15.167:0/4104956593 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:04.706 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.611+0000 7f5c337fe640 1 -- 172.21.15.167:0/4104956593 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c34199510 con 0x7f5c341050c0 2024-06-18T12:58:04.706 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.611+0000 7f5c337fe640 1 --2- 172.21.15.167:0/4104956593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c341050c0 0x7f5c34198fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5c24000c00 tx=0x7f5c240327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:04.706 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.611+0000 7f5c317fa640 1 -- 172.21.15.167:0/4104956593 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c24037040 con 0x7f5c341050c0 2024-06-18T12:58:04.706 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.611+0000 7f5c317fa640 1 -- 172.21.15.167:0/4104956593 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f5c24040070 con 0x7f5c341050c0 2024-06-18T12:58:04.706 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.611+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/4104956593 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c341997a0 con 0x7f5c341050c0 2024-06-18T12:58:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.611+0000 7f5c317fa640 1 -- 172.21.15.167:0/4104956593 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c2403bda0 con 0x7f5c341050c0 2024-06-18T12:58:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.611+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/4104956593 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c3419c490 con 0x7f5c341050c0 2024-06-18T12:58:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.612+0000 7f5c317fa640 1 -- 172.21.15.167:0/4104956593 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5c24045460 con 0x7f5c341050c0 2024-06-18T12:58:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.612+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/4104956593 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c34105540 con 0x7f5c341050c0 2024-06-18T12:58:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.612+0000 7f5c317fa640 1 --2- 172.21.15.167:0/4104956593 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f5c20038fc0 0x7f5c2003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.612+0000 7f5c317fa640 1 -- 172.21.15.167:0/4104956593 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5c2406c2a0 con 0x7f5c341050c0 2024-06-18T12:58:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.612+0000 7f5c2bfff640 1 --2- 172.21.15.167:0/4104956593 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f5c20038fc0 0x7f5c2003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.613+0000 7f5c2bfff640 1 --2- 172.21.15.167:0/4104956593 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f5c20038fc0 0x7f5c2003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0098c0 tx=0x7f5c1c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.617+0000 7f5c317fa640 1 -- 172.21.15.167:0/4104956593 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f5c24045850 con 0x7f5c341050c0 2024-06-18T12:58:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.811+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/4104956593 --> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f5c34101b20 con 0x7f5c20038fc0 2024-06-18T12:58:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.818+0000 7f5c317fa640 1 -- 172.21.15.167:0/4104956593 <== mgr.14118 v2:172.21.15.167:6800/1873453520 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5c34101b20 con 0x7f5c20038fc0 2024-06-18T12:58:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.820+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/4104956593 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f5c20038fc0 msgr2=0x7f5c2003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.820+0000 7f5c3a0d4640 1 --2- 172.21.15.167:0/4104956593 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f5c20038fc0 0x7f5c2003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0098c0 tx=0x7f5c1c006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:04.707 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.820+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/4104956593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c341050c0 msgr2=0x7f5c34198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:04.708 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.820+0000 7f5c3a0d4640 1 --2- 172.21.15.167:0/4104956593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c341050c0 0x7f5c34198fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5c24000c00 tx=0x7f5c240327b0 comp rx=0 tx=0).stop 2024-06-18T12:58:04.708 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.820+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/4104956593 shutdown_connections 2024-06-18T12:58:04.708 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.820+0000 7f5c3a0d4640 1 --2- 172.21.15.167:0/4104956593 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f5c20038fc0 0x7f5c2003b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:04.708 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.820+0000 7f5c3a0d4640 1 --2- 172.21.15.167:0/4104956593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c341050c0 0x7f5c34198fd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:04.708 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.820+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/4104956593 >> 172.21.15.167:0/4104956593 conn(0x7f5c341008b0 msgr2=0x7f5c34101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:04.708 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.820+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/4104956593 shutdown_connections 2024-06-18T12:58:04.708 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:03.820+0000 7f5c3a0d4640 1 -- 172.21.15.167:0/4104956593 wait complete. 2024-06-18T12:58:04.931 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:04 smithi167 ceph-mon[29496]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:58:04.932 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:04 smithi167 ceph-mon[29496]: Saving service mgr spec with placement count:2 2024-06-18T12:58:04.932 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:04 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' 2024-06-18T12:58:05.929 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.119+0000 7f003d51c640 1 Processor -- start 2024-06-18T12:58:05.929 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.119+0000 7f003d51c640 1 -- start start 2024-06-18T12:58:05.929 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.119+0000 7f003d51c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0038103fa0 0x7f00381043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:05.929 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.119+0000 7f003d51c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0038104970 con 0x7f0038103fa0 2024-06-18T12:58:05.929 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.120+0000 7f0036ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0038103fa0 0x7f00381043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:05.929 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.120+0000 7f0036ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0038103fa0 0x7f00381043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35506/0 (socket says 172.21.15.167:35506) 2024-06-18T12:58:05.929 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.120+0000 7f0036ffd640 1 -- 172.21.15.167:0/4207653020 learned_addr learned my addr 172.21.15.167:0/4207653020 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:05.929 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.120+0000 7f0036ffd640 1 -- 172.21.15.167:0/4207653020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00381051a0 con 0x7f0038103fa0 2024-06-18T12:58:05.929 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.121+0000 7f0036ffd640 1 --2- 172.21.15.167:0/4207653020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0038103fa0 0x7f00381043a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0028009820 tx=0x7f002802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba24bc409b0973f server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:05.929 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.121+0000 7f00367fc640 1 -- 172.21.15.167:0/4207653020 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0028037070 con 0x7f0038103fa0 2024-06-18T12:58:05.929 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.121+0000 7f00367fc640 1 -- 172.21.15.167:0/4207653020 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f00280328a0 con 0x7f0038103fa0 2024-06-18T12:58:05.930 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.121+0000 7f00367fc640 1 -- 172.21.15.167:0/4207653020 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0028030640 con 0x7f0038103fa0 2024-06-18T12:58:05.930 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.121+0000 7f003d51c640 1 -- 172.21.15.167:0/4207653020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0038103fa0 msgr2=0x7f00381043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:05.930 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.121+0000 7f003d51c640 1 --2- 172.21.15.167:0/4207653020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0038103fa0 0x7f00381043a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0028009820 tx=0x7f002802a270 comp rx=0 tx=0).stop 2024-06-18T12:58:05.930 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.121+0000 7f003d51c640 1 -- 172.21.15.167:0/4207653020 shutdown_connections 2024-06-18T12:58:05.930 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.122+0000 7f003d51c640 1 --2- 172.21.15.167:0/4207653020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0038103fa0 0x7f00381043a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:05.930 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.122+0000 7f003d51c640 1 -- 172.21.15.167:0/4207653020 >> 172.21.15.167:0/4207653020 conn(0x7f00380ff750 msgr2=0x7f0038101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:05.930 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.122+0000 7f003d51c640 1 -- 172.21.15.167:0/4207653020 shutdown_connections 2024-06-18T12:58:05.930 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.122+0000 7f003d51c640 1 -- 172.21.15.167:0/4207653020 wait complete. 2024-06-18T12:58:05.930 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.122+0000 7f003d51c640 1 Processor -- start 2024-06-18T12:58:05.930 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.122+0000 7f003d51c640 1 -- start start 2024-06-18T12:58:05.930 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.123+0000 7f003d51c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0038103fa0 0x7f003818f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:05.930 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.123+0000 7f003d51c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f00381083d0 con 0x7f0038103fa0 2024-06-18T12:58:05.930 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.123+0000 7f0036ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0038103fa0 0x7f003818f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:05.931 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.123+0000 7f0036ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0038103fa0 0x7f003818f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35512/0 (socket says 172.21.15.167:35512) 2024-06-18T12:58:05.931 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.123+0000 7f0036ffd640 1 -- 172.21.15.167:0/3854176361 learned_addr learned my addr 172.21.15.167:0/3854176361 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:05.931 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.123+0000 7f0036ffd640 1 -- 172.21.15.167:0/3854176361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003818f5d0 con 0x7f0038103fa0 2024-06-18T12:58:05.931 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.123+0000 7f0036ffd640 1 --2- 172.21.15.167:0/3854176361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0038103fa0 0x7f003818f090 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0028000c00 tx=0x7f00280327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:05.931 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.123+0000 7f0034ff9640 1 -- 172.21.15.167:0/3854176361 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0028037040 con 0x7f0038103fa0 2024-06-18T12:58:05.931 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.123+0000 7f0034ff9640 1 -- 172.21.15.167:0/3854176361 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f0028040070 con 0x7f0038103fa0 2024-06-18T12:58:05.931 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.123+0000 7f003d51c640 1 -- 172.21.15.167:0/3854176361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003818d7c0 con 0x7f0038103fa0 2024-06-18T12:58:05.931 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.123+0000 7f0034ff9640 1 -- 172.21.15.167:0/3854176361 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f002803bda0 con 0x7f0038103fa0 2024-06-18T12:58:05.931 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.123+0000 7f003d51c640 1 -- 172.21.15.167:0/3854176361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f003818dca0 con 0x7f0038103fa0 2024-06-18T12:58:05.931 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.124+0000 7f0034ff9640 1 -- 172.21.15.167:0/3854176361 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0028045460 con 0x7f0038103fa0 2024-06-18T12:58:05.931 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.124+0000 7f0034ff9640 1 --2- 172.21.15.167:0/3854176361 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f0024038f70 0x7f002403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:05.931 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.125+0000 7f0034ff9640 1 -- 172.21.15.167:0/3854176361 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f002806c220 con 0x7f0038103fa0 2024-06-18T12:58:05.932 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.125+0000 7f003d51c640 1 -- 172.21.15.167:0/3854176361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0038104420 con 0x7f0038103fa0 2024-06-18T12:58:05.932 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.125+0000 7f002e7fc640 1 --2- 172.21.15.167:0/3854176361 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f0024038f70 0x7f002403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:05.932 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.125+0000 7f002e7fc640 1 --2- 172.21.15.167:0/3854176361 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f0024038f70 0x7f002403b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f00200098c0 tx=0x7f0020006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:05.932 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.130+0000 7f0034ff9640 1 -- 172.21.15.167:0/3854176361 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f002802e200 con 0x7f0038103fa0 2024-06-18T12:58:05.932 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.310+0000 7f003d51c640 1 -- 172.21.15.167:0/3854176361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f0038192ba0 con 0x7f0038103fa0 2024-06-18T12:58:05.932 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.315+0000 7f0034ff9640 1 -- 172.21.15.167:0/3854176361 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f002803b430 con 0x7f0038103fa0 2024-06-18T12:58:05.932 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.318+0000 7f003d51c640 1 -- 172.21.15.167:0/3854176361 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f0024038f70 msgr2=0x7f002403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:05.932 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.318+0000 7f003d51c640 1 --2- 172.21.15.167:0/3854176361 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f0024038f70 0x7f002403b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f00200098c0 tx=0x7f0020006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:05.932 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.318+0000 7f003d51c640 1 -- 172.21.15.167:0/3854176361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0038103fa0 msgr2=0x7f003818f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:05.932 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.318+0000 7f003d51c640 1 --2- 172.21.15.167:0/3854176361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0038103fa0 0x7f003818f090 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0028000c00 tx=0x7f00280327b0 comp rx=0 tx=0).stop 2024-06-18T12:58:05.932 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.318+0000 7f003d51c640 1 -- 172.21.15.167:0/3854176361 shutdown_connections 2024-06-18T12:58:05.932 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.318+0000 7f003d51c640 1 --2- 172.21.15.167:0/3854176361 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f0024038f70 0x7f002403b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:05.932 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.318+0000 7f003d51c640 1 --2- 172.21.15.167:0/3854176361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0038103fa0 0x7f003818f090 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:05.932 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.318+0000 7f003d51c640 1 -- 172.21.15.167:0/3854176361 >> 172.21.15.167:0/3854176361 conn(0x7f00380ff750 msgr2=0x7f0038100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:05.933 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.318+0000 7f003d51c640 1 -- 172.21.15.167:0/3854176361 shutdown_connections 2024-06-18T12:58:05.933 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:05.318+0000 7f003d51c640 1 -- 172.21.15.167:0/3854176361 wait complete. 2024-06-18T12:58:06.527 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3854176361' entity='client.admin' 2024-06-18T12:58:06.527 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:06 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' 2024-06-18T12:58:07.044 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.324+0000 7f9d670e7640 1 Processor -- start 2024-06-18T12:58:07.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.324+0000 7f9d670e7640 1 -- start start 2024-06-18T12:58:07.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.324+0000 7f9d670e7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d60105110 0x7f9d60105510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:07.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.324+0000 7f9d670e7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60105ae0 con 0x7f9d60105110 2024-06-18T12:58:07.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.324+0000 7f9d64e83640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d60105110 0x7f9d60105510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:07.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.324+0000 7f9d64e83640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d60105110 0x7f9d60105510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35522/0 (socket says 172.21.15.167:35522) 2024-06-18T12:58:07.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.324+0000 7f9d64e83640 1 -- 172.21.15.167:0/3563002156 learned_addr learned my addr 172.21.15.167:0/3563002156 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:07.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.324+0000 7f9d64e83640 1 -- 172.21.15.167:0/3563002156 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d601062c0 con 0x7f9d60105110 2024-06-18T12:58:07.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.325+0000 7f9d64e83640 1 --2- 172.21.15.167:0/3563002156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d60105110 0x7f9d60105510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009820 tx=0x7f9d5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d348a0c03d44625c server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:07.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.325+0000 7f9d5f7fe640 1 -- 172.21.15.167:0/3563002156 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d54037070 con 0x7f9d60105110 2024-06-18T12:58:07.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.325+0000 7f9d5f7fe640 1 -- 172.21.15.167:0/3563002156 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f9d540328c0 con 0x7f9d60105110 2024-06-18T12:58:07.045 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.325+0000 7f9d5f7fe640 1 -- 172.21.15.167:0/3563002156 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d54030600 con 0x7f9d60105110 2024-06-18T12:58:07.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.326+0000 7f9d670e7640 1 -- 172.21.15.167:0/3563002156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d60105110 msgr2=0x7f9d60105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:07.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.326+0000 7f9d670e7640 1 --2- 172.21.15.167:0/3563002156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d60105110 0x7f9d60105510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009820 tx=0x7f9d5402a270 comp rx=0 tx=0).stop 2024-06-18T12:58:07.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.326+0000 7f9d670e7640 1 -- 172.21.15.167:0/3563002156 shutdown_connections 2024-06-18T12:58:07.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.326+0000 7f9d670e7640 1 --2- 172.21.15.167:0/3563002156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d60105110 0x7f9d60105510 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:07.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.326+0000 7f9d670e7640 1 -- 172.21.15.167:0/3563002156 >> 172.21.15.167:0/3563002156 conn(0x7f9d601008a0 msgr2=0x7f9d60102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:07.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.326+0000 7f9d670e7640 1 -- 172.21.15.167:0/3563002156 shutdown_connections 2024-06-18T12:58:07.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.326+0000 7f9d670e7640 1 -- 172.21.15.167:0/3563002156 wait complete. 2024-06-18T12:58:07.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.326+0000 7f9d670e7640 1 Processor -- start 2024-06-18T12:58:07.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.326+0000 7f9d670e7640 1 -- start start 2024-06-18T12:58:07.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.327+0000 7f9d670e7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d60105110 0x7f9d60198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:07.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.327+0000 7f9d670e7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60109670 con 0x7f9d60105110 2024-06-18T12:58:07.046 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.327+0000 7f9d64e83640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d60105110 0x7f9d60198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:07.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.327+0000 7f9d64e83640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d60105110 0x7f9d60198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35538/0 (socket says 172.21.15.167:35538) 2024-06-18T12:58:07.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.327+0000 7f9d64e83640 1 -- 172.21.15.167:0/93537422 learned_addr learned my addr 172.21.15.167:0/93537422 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:07.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.327+0000 7f9d64e83640 1 -- 172.21.15.167:0/93537422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d60199480 con 0x7f9d60105110 2024-06-18T12:58:07.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.327+0000 7f9d64e83640 1 --2- 172.21.15.167:0/93537422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d60105110 0x7f9d60198f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009940 tx=0x7f9d540327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:07.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.327+0000 7f9d5dffb640 1 -- 172.21.15.167:0/93537422 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d54037040 con 0x7f9d60105110 2024-06-18T12:58:07.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.327+0000 7f9d5dffb640 1 -- 172.21.15.167:0/93537422 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f9d54040070 con 0x7f9d60105110 2024-06-18T12:58:07.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.327+0000 7f9d5dffb640 1 -- 172.21.15.167:0/93537422 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d5403bd50 con 0x7f9d60105110 2024-06-18T12:58:07.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.327+0000 7f9d670e7640 1 -- 172.21.15.167:0/93537422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d60199710 con 0x7f9d60105110 2024-06-18T12:58:07.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.327+0000 7f9d670e7640 1 -- 172.21.15.167:0/93537422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d6019c400 con 0x7f9d60105110 2024-06-18T12:58:07.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.328+0000 7f9d5dffb640 1 -- 172.21.15.167:0/93537422 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9d54030c90 con 0x7f9d60105110 2024-06-18T12:58:07.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.328+0000 7f9d670e7640 1 -- 172.21.15.167:0/93537422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d60105510 con 0x7f9d60105110 2024-06-18T12:58:07.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.329+0000 7f9d5dffb640 1 --2- 172.21.15.167:0/93537422 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f9d50038fc0 0x7f9d5003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:07.047 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.329+0000 7f9d5dffb640 1 -- 172.21.15.167:0/93537422 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9d5406c370 con 0x7f9d60105110 2024-06-18T12:58:07.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.329+0000 7f9d5ffff640 1 --2- 172.21.15.167:0/93537422 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f9d50038fc0 0x7f9d5003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:07.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.329+0000 7f9d5ffff640 1 --2- 172.21.15.167:0/93537422 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f9d50038fc0 0x7f9d5003b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0098c0 tx=0x7f9d4c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:07.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.333+0000 7f9d5dffb640 1 -- 172.21.15.167:0/93537422 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f9d5403ae90 con 0x7f9d60105110 2024-06-18T12:58:07.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.527+0000 7f9d670e7640 1 -- 172.21.15.167:0/93537422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f9d6019c870 con 0x7f9d60105110 2024-06-18T12:58:07.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.530+0000 7f9d5dffb640 1 -- 172.21.15.167:0/93537422 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f9d54045980 con 0x7f9d60105110 2024-06-18T12:58:07.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.533+0000 7f9d670e7640 1 -- 172.21.15.167:0/93537422 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f9d50038fc0 msgr2=0x7f9d5003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:07.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.533+0000 7f9d670e7640 1 --2- 172.21.15.167:0/93537422 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f9d50038fc0 0x7f9d5003b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0098c0 tx=0x7f9d4c006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:07.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.533+0000 7f9d670e7640 1 -- 172.21.15.167:0/93537422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d60105110 msgr2=0x7f9d60198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:07.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.533+0000 7f9d670e7640 1 --2- 172.21.15.167:0/93537422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d60105110 0x7f9d60198f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009940 tx=0x7f9d540327b0 comp rx=0 tx=0).stop 2024-06-18T12:58:07.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.533+0000 7f9d670e7640 1 -- 172.21.15.167:0/93537422 shutdown_connections 2024-06-18T12:58:07.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.533+0000 7f9d670e7640 1 --2- 172.21.15.167:0/93537422 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f9d50038fc0 0x7f9d5003b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:07.048 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.533+0000 7f9d670e7640 1 --2- 172.21.15.167:0/93537422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d60105110 0x7f9d60198f40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:07.049 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.533+0000 7f9d670e7640 1 -- 172.21.15.167:0/93537422 >> 172.21.15.167:0/93537422 conn(0x7f9d601008a0 msgr2=0x7f9d60101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:07.049 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.533+0000 7f9d670e7640 1 -- 172.21.15.167:0/93537422 shutdown_connections 2024-06-18T12:58:07.049 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:06.533+0000 7f9d670e7640 1 -- 172.21.15.167:0/93537422 wait complete. 2024-06-18T12:58:07.049 INFO:teuthology.orchestra.run.smithi167.stdout:Enabling the dashboard module... 2024-06-18T12:58:07.684 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:07 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' 2024-06-18T12:58:07.684 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/93537422' entity='client.admin' 2024-06-18T12:58:08.716 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2707107032' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-18T12:58:08.716 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:08 smithi167 ceph-mon[29496]: from='mgr.14118 172.21.15.167:0/1541006089' entity='mgr.y' 2024-06-18T12:58:08.717 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:08 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ignoring --setuser ceph since I am not root 2024-06-18T12:58:08.717 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:08 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ignoring --setgroup ceph since I am not root 2024-06-18T12:58:08.717 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:08 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:08.612+0000 7f12facce040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-18T12:58:08.976 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:08 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:08.767+0000 7f12facce040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-18T12:58:08.976 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:08 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:08.842+0000 7f12facce040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-18T12:58:09.088 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.442+0000 7f462dc4c640 1 Processor -- start 2024-06-18T12:58:09.088 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.443+0000 7f462dc4c640 1 -- start start 2024-06-18T12:58:09.088 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.443+0000 7f462dc4c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4628105bf0 0x7f4628107fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:09.089 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.443+0000 7f462dc4c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4628078df0 con 0x7f4628105bf0 2024-06-18T12:58:09.089 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.443+0000 7f46277fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4628105bf0 0x7f4628107fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:09.089 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.443+0000 7f46277fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4628105bf0 0x7f4628107fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35548/0 (socket says 172.21.15.167:35548) 2024-06-18T12:58:09.089 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.443+0000 7f46277fe640 1 -- 172.21.15.167:0/717416225 learned_addr learned my addr 172.21.15.167:0/717416225 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:09.089 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.443+0000 7f46277fe640 1 -- 172.21.15.167:0/717416225 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4628108520 con 0x7f4628105bf0 2024-06-18T12:58:09.089 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.444+0000 7f46277fe640 1 --2- 172.21.15.167:0/717416225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4628105bf0 0x7f4628107fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4618009820 tx=0x7f461802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f6c21d60acae53 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:09.089 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.444+0000 7f46267fc640 1 -- 172.21.15.167:0/717416225 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4618037070 con 0x7f4628105bf0 2024-06-18T12:58:09.089 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.444+0000 7f46267fc640 1 -- 172.21.15.167:0/717416225 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f46180328c0 con 0x7f4628105bf0 2024-06-18T12:58:09.089 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.444+0000 7f46267fc640 1 -- 172.21.15.167:0/717416225 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4618030600 con 0x7f4628105bf0 2024-06-18T12:58:09.089 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.445+0000 7f462dc4c640 1 -- 172.21.15.167:0/717416225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4628105bf0 msgr2=0x7f4628107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:09.089 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.445+0000 7f462dc4c640 1 --2- 172.21.15.167:0/717416225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4628105bf0 0x7f4628107fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4618009820 tx=0x7f461802a270 comp rx=0 tx=0).stop 2024-06-18T12:58:09.089 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.445+0000 7f462dc4c640 1 -- 172.21.15.167:0/717416225 shutdown_connections 2024-06-18T12:58:09.089 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.445+0000 7f462dc4c640 1 --2- 172.21.15.167:0/717416225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4628105bf0 0x7f4628107fe0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:09.089 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.445+0000 7f462dc4c640 1 -- 172.21.15.167:0/717416225 >> 172.21.15.167:0/717416225 conn(0x7f46280ff790 msgr2=0x7f4628101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:09.090 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.445+0000 7f462dc4c640 1 -- 172.21.15.167:0/717416225 shutdown_connections 2024-06-18T12:58:09.090 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.445+0000 7f462dc4c640 1 -- 172.21.15.167:0/717416225 wait complete. 2024-06-18T12:58:09.090 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.445+0000 7f462dc4c640 1 Processor -- start 2024-06-18T12:58:09.090 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.446+0000 7f462dc4c640 1 -- start start 2024-06-18T12:58:09.090 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.446+0000 7f462dc4c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4628105bf0 0x7f4628199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:09.090 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.446+0000 7f462dc4c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4628109480 con 0x7f4628105bf0 2024-06-18T12:58:09.090 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.446+0000 7f46277fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4628105bf0 0x7f4628199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:09.090 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.446+0000 7f46277fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4628105bf0 0x7f4628199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35564/0 (socket says 172.21.15.167:35564) 2024-06-18T12:58:09.090 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.446+0000 7f46277fe640 1 -- 172.21.15.167:0/2707107032 learned_addr learned my addr 172.21.15.167:0/2707107032 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:09.090 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.446+0000 7f46277fe640 1 -- 172.21.15.167:0/2707107032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46281995a0 con 0x7f4628105bf0 2024-06-18T12:58:09.090 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.446+0000 7f46277fe640 1 --2- 172.21.15.167:0/2707107032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4628105bf0 0x7f4628199060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f46180097f0 tx=0x7f4618030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:09.090 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.446+0000 7f4624ff9640 1 -- 172.21.15.167:0/2707107032 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4618037040 con 0x7f4628105bf0 2024-06-18T12:58:09.091 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.446+0000 7f462dc4c640 1 -- 172.21.15.167:0/2707107032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4628199830 con 0x7f4628105bf0 2024-06-18T12:58:09.091 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.446+0000 7f4624ff9640 1 -- 172.21.15.167:0/2707107032 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f4618040070 con 0x7f4628105bf0 2024-06-18T12:58:09.091 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.446+0000 7f4624ff9640 1 -- 172.21.15.167:0/2707107032 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f461803bdd0 con 0x7f4628105bf0 2024-06-18T12:58:09.091 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.446+0000 7f462dc4c640 1 -- 172.21.15.167:0/2707107032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462819c520 con 0x7f4628105bf0 2024-06-18T12:58:09.091 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.447+0000 7f4624ff9640 1 -- 172.21.15.167:0/2707107032 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f461803b430 con 0x7f4628105bf0 2024-06-18T12:58:09.091 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.447+0000 7f462dc4c640 1 -- 172.21.15.167:0/2707107032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4628105fd0 con 0x7f4628105bf0 2024-06-18T12:58:09.091 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.448+0000 7f4624ff9640 1 --2- 172.21.15.167:0/2707107032 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f4614038f20 0x7f461403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:09.091 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.448+0000 7f4624ff9640 1 -- 172.21.15.167:0/2707107032 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f461806c2e0 con 0x7f4628105bf0 2024-06-18T12:58:09.091 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.448+0000 7f4626ffd640 1 --2- 172.21.15.167:0/2707107032 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f4614038f20 0x7f461403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:09.091 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.448+0000 7f4626ffd640 1 --2- 172.21.15.167:0/2707107032 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f4614038f20 0x7f461403b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f46100098c0 tx=0x7f4610006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:09.091 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.452+0000 7f4624ff9640 1 -- 172.21.15.167:0/2707107032 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f46180371e0 con 0x7f4628105bf0 2024-06-18T12:58:09.091 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:07.684+0000 7f462dc4c640 1 -- 172.21.15.167:0/2707107032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f462819c870 con 0x7f4628105bf0 2024-06-18T12:58:09.091 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:08.459+0000 7f4624ff9640 1 -- 172.21.15.167:0/2707107032 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f461803b720 con 0x7f4628105bf0 2024-06-18T12:58:09.091 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:08.461+0000 7f4624ff9640 1 -- 172.21.15.167:0/2707107032 <== mon.0 v2:172.21.15.167:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f461804acf0 con 0x7f4628105bf0 2024-06-18T12:58:09.092 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:08.462+0000 7f462dc4c640 1 -- 172.21.15.167:0/2707107032 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f4614038f20 msgr2=0x7f461403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:09.092 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:08.462+0000 7f462dc4c640 1 --2- 172.21.15.167:0/2707107032 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f4614038f20 0x7f461403b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f46100098c0 tx=0x7f4610006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:09.092 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:08.462+0000 7f462dc4c640 1 -- 172.21.15.167:0/2707107032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4628105bf0 msgr2=0x7f4628199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:09.092 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:08.462+0000 7f462dc4c640 1 --2- 172.21.15.167:0/2707107032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4628105bf0 0x7f4628199060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f46180097f0 tx=0x7f4618030b30 comp rx=0 tx=0).stop 2024-06-18T12:58:09.092 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:08.463+0000 7f462dc4c640 1 -- 172.21.15.167:0/2707107032 shutdown_connections 2024-06-18T12:58:09.092 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:08.463+0000 7f462dc4c640 1 --2- 172.21.15.167:0/2707107032 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f4614038f20 0x7f461403b3e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:09.092 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:08.463+0000 7f462dc4c640 1 --2- 172.21.15.167:0/2707107032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4628105bf0 0x7f4628199060 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:09.092 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:08.463+0000 7f462dc4c640 1 -- 172.21.15.167:0/2707107032 >> 172.21.15.167:0/2707107032 conn(0x7f46280ff790 msgr2=0x7f4628100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:09.092 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:08.463+0000 7f462dc4c640 1 -- 172.21.15.167:0/2707107032 shutdown_connections 2024-06-18T12:58:09.092 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:08.463+0000 7f462dc4c640 1 -- 172.21.15.167:0/2707107032 wait complete. 2024-06-18T12:58:09.384 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:09.097+0000 7f12facce040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-18T12:58:09.384 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:09.161+0000 7f12facce040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-18T12:58:09.672 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2707107032' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-18T12:58:09.673 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:09 smithi167 ceph-mon[29496]: mgrmap e9: y(active, since 17s) 2024-06-18T12:58:10.278 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout { 2024-06-18T12:58:10.278 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-18T12:58:10.278 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-18T12:58:10.279 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-06-18T12:58:10.279 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-18T12:58:10.279 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout } 2024-06-18T12:58:10.279 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.466+0000 7f98463c0640 1 Processor -- start 2024-06-18T12:58:10.279 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.466+0000 7f98463c0640 1 -- start start 2024-06-18T12:58:10.279 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.466+0000 7f98463c0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9840104040 0x7f9840104440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:10.279 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.466+0000 7f98463c0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9840104a10 con 0x7f9840104040 2024-06-18T12:58:10.279 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.467+0000 7f983ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9840104040 0x7f9840104440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:10.279 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.467+0000 7f983ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9840104040 0x7f9840104440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35598/0 (socket says 172.21.15.167:35598) 2024-06-18T12:58:10.279 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.467+0000 7f983ffff640 1 -- 172.21.15.167:0/192696623 learned_addr learned my addr 172.21.15.167:0/192696623 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:10.279 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.467+0000 7f983ffff640 1 -- 172.21.15.167:0/192696623 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9840105240 con 0x7f9840104040 2024-06-18T12:58:10.279 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.468+0000 7f983ffff640 1 --2- 172.21.15.167:0/192696623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9840104040 0x7f9840104440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9830009820 tx=0x7f983002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191be26f06ee9c7e server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:10.279 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.468+0000 7f983effd640 1 -- 172.21.15.167:0/192696623 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9830037070 con 0x7f9840104040 2024-06-18T12:58:10.280 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.468+0000 7f983effd640 1 -- 172.21.15.167:0/192696623 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f98300328c0 con 0x7f9840104040 2024-06-18T12:58:10.280 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.468+0000 7f983effd640 1 -- 172.21.15.167:0/192696623 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9830030600 con 0x7f9840104040 2024-06-18T12:58:10.280 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.469+0000 7f98463c0640 1 -- 172.21.15.167:0/192696623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9840104040 msgr2=0x7f9840104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:10.280 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.469+0000 7f98463c0640 1 --2- 172.21.15.167:0/192696623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9840104040 0x7f9840104440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9830009820 tx=0x7f983002a270 comp rx=0 tx=0).stop 2024-06-18T12:58:10.280 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.469+0000 7f98463c0640 1 -- 172.21.15.167:0/192696623 shutdown_connections 2024-06-18T12:58:10.280 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.469+0000 7f98463c0640 1 --2- 172.21.15.167:0/192696623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9840104040 0x7f9840104440 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:10.280 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.469+0000 7f98463c0640 1 -- 172.21.15.167:0/192696623 >> 172.21.15.167:0/192696623 conn(0x7f98400ff790 msgr2=0x7f9840101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:10.280 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.469+0000 7f98463c0640 1 -- 172.21.15.167:0/192696623 shutdown_connections 2024-06-18T12:58:10.280 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.469+0000 7f98463c0640 1 -- 172.21.15.167:0/192696623 wait complete. 2024-06-18T12:58:10.280 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.469+0000 7f98463c0640 1 Processor -- start 2024-06-18T12:58:10.281 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.469+0000 7f98463c0640 1 -- start start 2024-06-18T12:58:10.281 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.470+0000 7f98463c0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9840104040 0x7f9840199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:10.281 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.470+0000 7f98463c0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f98401085f0 con 0x7f9840104040 2024-06-18T12:58:10.281 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.470+0000 7f983ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9840104040 0x7f9840199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:10.281 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.470+0000 7f983ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9840104040 0x7f9840199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35614/0 (socket says 172.21.15.167:35614) 2024-06-18T12:58:10.281 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.470+0000 7f983ffff640 1 -- 172.21.15.167:0/1502077056 learned_addr learned my addr 172.21.15.167:0/1502077056 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:10.281 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.470+0000 7f983ffff640 1 -- 172.21.15.167:0/1502077056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98401995b0 con 0x7f9840104040 2024-06-18T12:58:10.281 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.470+0000 7f983ffff640 1 --2- 172.21.15.167:0/1502077056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9840104040 0x7f9840199070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9830009940 tx=0x7f98300327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:10.281 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.470+0000 7f983d7fa640 1 -- 172.21.15.167:0/1502077056 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9830037040 con 0x7f9840104040 2024-06-18T12:58:10.281 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.470+0000 7f983d7fa640 1 -- 172.21.15.167:0/1502077056 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f9830040070 con 0x7f9840104040 2024-06-18T12:58:10.281 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.470+0000 7f98463c0640 1 -- 172.21.15.167:0/1502077056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9840199840 con 0x7f9840104040 2024-06-18T12:58:10.281 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.471+0000 7f98463c0640 1 -- 172.21.15.167:0/1502077056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984019c530 con 0x7f9840104040 2024-06-18T12:58:10.281 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.471+0000 7f983d7fa640 1 -- 172.21.15.167:0/1502077056 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f983003bd30 con 0x7f9840104040 2024-06-18T12:58:10.282 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.471+0000 7f983d7fa640 1 -- 172.21.15.167:0/1502077056 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f98300454a0 con 0x7f9840104040 2024-06-18T12:58:10.282 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.472+0000 7f98463c0640 1 -- 172.21.15.167:0/1502077056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98200050b0 con 0x7f9840104040 2024-06-18T12:58:10.282 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.472+0000 7f983d7fa640 1 --2- 172.21.15.167:0/1502077056 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f982c038fc0 0x7f982c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:10.282 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.472+0000 7f983d7fa640 1 -- 172.21.15.167:0/1502077056 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f983006c340 con 0x7f9840104040 2024-06-18T12:58:10.282 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.472+0000 7f983f7fe640 1 -- 172.21.15.167:0/1502077056 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f982c038fc0 msgr2=0x7f982c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/1873453520 2024-06-18T12:58:10.282 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.472+0000 7f983f7fe640 1 --2- 172.21.15.167:0/1502077056 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f982c038fc0 0x7f982c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-18T12:58:10.282 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.477+0000 7f983d7fa640 1 -- 172.21.15.167:0/1502077056 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f9830045930 con 0x7f9840104040 2024-06-18T12:58:10.282 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.673+0000 7f983f7fe640 1 -- 172.21.15.167:0/1502077056 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f982c038fc0 msgr2=0x7f982c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/1873453520 2024-06-18T12:58:10.282 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.673+0000 7f983f7fe640 1 --2- 172.21.15.167:0/1502077056 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f982c038fc0 0x7f982c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-18T12:58:10.282 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.708+0000 7f98463c0640 1 -- 172.21.15.167:0/1502077056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9820005bb0 con 0x7f9840104040 2024-06-18T12:58:10.282 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.708+0000 7f983d7fa640 1 -- 172.21.15.167:0/1502077056 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f98300443e0 con 0x7f9840104040 2024-06-18T12:58:10.282 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.710+0000 7f98463c0640 1 -- 172.21.15.167:0/1502077056 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f982c038fc0 msgr2=0x7f982c03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-18T12:58:10.282 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.710+0000 7f98463c0640 1 --2- 172.21.15.167:0/1502077056 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f982c038fc0 0x7f982c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:10.283 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.710+0000 7f98463c0640 1 -- 172.21.15.167:0/1502077056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9840104040 msgr2=0x7f9840199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:10.283 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.710+0000 7f98463c0640 1 --2- 172.21.15.167:0/1502077056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9840104040 0x7f9840199070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9830009940 tx=0x7f98300327b0 comp rx=0 tx=0).stop 2024-06-18T12:58:10.283 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.711+0000 7f98463c0640 1 -- 172.21.15.167:0/1502077056 shutdown_connections 2024-06-18T12:58:10.283 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.711+0000 7f98463c0640 1 --2- 172.21.15.167:0/1502077056 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f982c038fc0 0x7f982c03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:10.283 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.711+0000 7f98463c0640 1 --2- 172.21.15.167:0/1502077056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9840104040 0x7f9840199070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:10.283 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.711+0000 7f98463c0640 1 -- 172.21.15.167:0/1502077056 >> 172.21.15.167:0/1502077056 conn(0x7f98400ff790 msgr2=0x7f98401006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:10.283 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.711+0000 7f98463c0640 1 -- 172.21.15.167:0/1502077056 shutdown_connections 2024-06-18T12:58:10.283 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:09.711+0000 7f98463c0640 1 -- 172.21.15.167:0/1502077056 wait complete. 2024-06-18T12:58:10.283 INFO:teuthology.orchestra.run.smithi167.stdout:Waiting for the mgr to restart... 2024-06-18T12:58:10.283 INFO:teuthology.orchestra.run.smithi167.stdout:Waiting for mgr epoch 9... 2024-06-18T12:58:10.324 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:10 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:10.140+0000 7f12facce040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-18T12:58:10.325 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:10 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:10.260+0000 7f12facce040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-18T12:58:10.325 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:10 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:10.323+0000 7f12facce040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-18T12:58:10.610 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1502077056' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-18T12:58:10.611 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:10 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:10.547+0000 7f12facce040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-18T12:58:10.911 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:10 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:10.610+0000 7f12facce040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-18T12:58:10.912 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:10 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:10.682+0000 7f12facce040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-18T12:58:10.912 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:10 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:10.836+0000 7f12facce040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-18T12:58:11.582 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:11 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:11.258+0000 7f12facce040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-18T12:58:11.582 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:11 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:11.581+0000 7f12facce040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-18T12:58:11.859 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:11 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:11.652+0000 7f12facce040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-18T12:58:11.859 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:11 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:11.812+0000 7f12facce040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-18T12:58:12.410 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:12 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:12.147+0000 7f12facce040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-18T12:58:12.411 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:12 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:12.213+0000 7f12facce040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-18T12:58:12.721 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:12 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:12.410+0000 7f12facce040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-18T12:58:12.721 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:12 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:12.564+0000 7f12facce040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-18T12:58:12.721 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:12 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:12.632+0000 7f12facce040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-18T12:58:12.721 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:12 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:12.720+0000 7f12facce040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-18T12:58:12.976 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:12 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:12.783+0000 7f12facce040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-18T12:58:13.642 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:13.326+0000 7f12facce040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-18T12:58:13.642 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:13.487+0000 7f12facce040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-18T12:58:13.642 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-mon[29496]: Active manager daemon y restarted 2024-06-18T12:58:13.642 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-mon[29496]: Activating manager daemon y 2024-06-18T12:58:13.642 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-mon[29496]: osdmap e3: 0 total, 0 up, 0 in 2024-06-18T12:58:13.642 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-mon[29496]: mgrmap e10: y(active, starting, since 0.00319309s) 2024-06-18T12:58:13.642 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T12:58:13.642 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-18T12:58:13.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-18T12:58:13.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-18T12:58:13.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-18T12:58:13.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-mon[29496]: Manager daemon y is now available 2024-06-18T12:58:13.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:58:13.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-18T12:58:13.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-18T12:58:15.166 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:14.828+0000 7f1261296640 -1 log_channel(cephadm) log [ERR] : [18/Jun/2024:12:58:14] ENGINE Error in HTTPServer.serve 2024-06-18T12:58:15.167 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: Traceback (most recent call last): 2024-06-18T12:58:15.167 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-18T12:58:15.167 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: self._connections.run(self.expiration_interval) 2024-06-18T12:58:15.167 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-18T12:58:15.167 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: self._run(expiration_interval) 2024-06-18T12:58:15.167 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-18T12:58:15.167 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: new_conn = self._from_server_socket(self.server.socket) 2024-06-18T12:58:15.167 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-18T12:58:15.167 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-18T12:58:15.167 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-18T12:58:15.167 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: s = self.context.wrap_socket( 2024-06-18T12:58:15.167 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-18T12:58:15.167 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: return self.sslsocket_class._create( 2024-06-18T12:58:15.167 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-18T12:58:15.168 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: self.do_handshake() 2024-06-18T12:58:15.168 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-18T12:58:15.168 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: self._sslobj.do_handshake() 2024-06-18T12:58:15.168 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-18T12:58:15.168 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 12:58:14 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T12:58:15.175 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout { 2024-06-18T12:58:15.175 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-18T12:58:15.175 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-18T12:58:15.175 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout } 2024-06-18T12:58:15.175 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.703+0000 7f60ba207640 1 Processor -- start 2024-06-18T12:58:15.175 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.704+0000 7f60ba207640 1 -- start start 2024-06-18T12:58:15.175 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.704+0000 7f60ba207640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b4105ba0 0x7f60b4107f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:15.175 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.704+0000 7f60ba207640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60b4079640 con 0x7f60b4105ba0 2024-06-18T12:58:15.175 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.704+0000 7f60b37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b4105ba0 0x7f60b4107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:15.175 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.704+0000 7f60b37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b4105ba0 0x7f60b4107f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35620/0 (socket says 172.21.15.167:35620) 2024-06-18T12:58:15.176 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.704+0000 7f60b37fe640 1 -- 172.21.15.167:0/375587211 learned_addr learned my addr 172.21.15.167:0/375587211 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:15.176 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.704+0000 7f60b37fe640 1 -- 172.21.15.167:0/375587211 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b41084d0 con 0x7f60b4105ba0 2024-06-18T12:58:15.176 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.705+0000 7f60b37fe640 1 --2- 172.21.15.167:0/375587211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b4105ba0 0x7f60b4107f90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009820 tx=0x7f60a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100ad8e0b01de480 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:15.176 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.705+0000 7f60b27fc640 1 -- 172.21.15.167:0/375587211 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60a4037070 con 0x7f60b4105ba0 2024-06-18T12:58:15.176 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.705+0000 7f60b27fc640 1 -- 172.21.15.167:0/375587211 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f60a40328c0 con 0x7f60b4105ba0 2024-06-18T12:58:15.176 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.705+0000 7f60b27fc640 1 -- 172.21.15.167:0/375587211 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60a4030600 con 0x7f60b4105ba0 2024-06-18T12:58:15.176 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.706+0000 7f60ba207640 1 -- 172.21.15.167:0/375587211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b4105ba0 msgr2=0x7f60b4107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:15.176 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.706+0000 7f60ba207640 1 --2- 172.21.15.167:0/375587211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b4105ba0 0x7f60b4107f90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009820 tx=0x7f60a402a270 comp rx=0 tx=0).stop 2024-06-18T12:58:15.176 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.706+0000 7f60ba207640 1 -- 172.21.15.167:0/375587211 shutdown_connections 2024-06-18T12:58:15.176 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.706+0000 7f60ba207640 1 --2- 172.21.15.167:0/375587211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b4105ba0 0x7f60b4107f90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:15.176 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.706+0000 7f60ba207640 1 -- 172.21.15.167:0/375587211 >> 172.21.15.167:0/375587211 conn(0x7f60b40ff780 msgr2=0x7f60b4101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:15.176 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.706+0000 7f60ba207640 1 -- 172.21.15.167:0/375587211 shutdown_connections 2024-06-18T12:58:15.176 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.706+0000 7f60ba207640 1 -- 172.21.15.167:0/375587211 wait complete. 2024-06-18T12:58:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.707+0000 7f60ba207640 1 Processor -- start 2024-06-18T12:58:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.707+0000 7f60ba207640 1 -- start start 2024-06-18T12:58:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.707+0000 7f60ba207640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b4105ba0 0x7f60b419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.707+0000 7f60ba207640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60b41093d0 con 0x7f60b4105ba0 2024-06-18T12:58:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.707+0000 7f60b37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b4105ba0 0x7f60b419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.707+0000 7f60b37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b4105ba0 0x7f60b419b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35634/0 (socket says 172.21.15.167:35634) 2024-06-18T12:58:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.707+0000 7f60b37fe640 1 -- 172.21.15.167:0/147700249 learned_addr learned my addr 172.21.15.167:0/147700249 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.708+0000 7f60b37fe640 1 -- 172.21.15.167:0/147700249 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b419b650 con 0x7f60b4105ba0 2024-06-18T12:58:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.708+0000 7f60b37fe640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b4105ba0 0x7f60b419b110 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f60a40327b0 tx=0x7f60a40327e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.708+0000 7f60b0ff9640 1 -- 172.21.15.167:0/147700249 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60a4037040 con 0x7f60b4105ba0 2024-06-18T12:58:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.708+0000 7f60b0ff9640 1 -- 172.21.15.167:0/147700249 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f60a4040070 con 0x7f60b4105ba0 2024-06-18T12:58:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.708+0000 7f60b0ff9640 1 -- 172.21.15.167:0/147700249 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60a403be50 con 0x7f60b4105ba0 2024-06-18T12:58:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.708+0000 7f60ba207640 1 -- 172.21.15.167:0/147700249 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b419b8e0 con 0x7f60b4105ba0 2024-06-18T12:58:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.708+0000 7f60ba207640 1 -- 172.21.15.167:0/147700249 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b419e5d0 con 0x7f60b4105ba0 2024-06-18T12:58:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.709+0000 7f60b0ff9640 1 -- 172.21.15.167:0/147700249 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f60a4045460 con 0x7f60b4105ba0 2024-06-18T12:58:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.709+0000 7f60b0ff9640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f609c0391c0 0x7f609c03b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.709+0000 7f60b0ff9640 1 -- 172.21.15.167:0/147700249 --> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f60a403be50 con 0x7f609c0391c0 2024-06-18T12:58:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.710+0000 7f60b0ff9640 1 -- 172.21.15.167:0/147700249 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f60a406c420 con 0x7f60b4105ba0 2024-06-18T12:58:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.710+0000 7f60b2ffd640 1 -- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f609c0391c0 msgr2=0x7f609c03b680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/1873453520 2024-06-18T12:58:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.710+0000 7f60b2ffd640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f609c0391c0 0x7f609c03b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-18T12:58:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.911+0000 7f60b2ffd640 1 -- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f609c0391c0 msgr2=0x7f609c03b680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/1873453520 2024-06-18T12:58:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:10.911+0000 7f60b2ffd640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f609c0391c0 0x7f609c03b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-18T12:58:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:11.312+0000 7f60b2ffd640 1 -- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f609c0391c0 msgr2=0x7f609c03b680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/1873453520 2024-06-18T12:58:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:11.312+0000 7f60b2ffd640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f609c0391c0 0x7f609c03b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-18T12:58:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:12.113+0000 7f60b2ffd640 1 -- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f609c0391c0 msgr2=0x7f609c03b680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/1873453520 2024-06-18T12:58:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:12.113+0000 7f60b2ffd640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f609c0391c0 0x7f609c03b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-18T12:58:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:13.490+0000 7f60b0ff9640 1 -- 172.21.15.167:0/147700249 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f60a4030ce0 con 0x7f60b4105ba0 2024-06-18T12:58:15.179 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:13.491+0000 7f60b0ff9640 1 -- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f609c0391c0 msgr2=0x7f609c03b680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-18T12:58:15.179 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:13.491+0000 7f60b0ff9640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f609c0391c0 0x7f609c03b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:15.179 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.492+0000 7f60b0ff9640 1 -- 172.21.15.167:0/147700249 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f60a4045750 con 0x7f60b4105ba0 2024-06-18T12:58:15.179 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.492+0000 7f60b0ff9640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f609c03cad0 0x7f609c03eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:15.179 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.492+0000 7f60b0ff9640 1 -- 172.21.15.167:0/147700249 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f60a4039510 con 0x7f609c03cad0 2024-06-18T12:58:15.179 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.493+0000 7f60b2ffd640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f609c03cad0 0x7f609c03eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:15.179 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.493+0000 7f60b2ffd640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f609c03cad0 0x7f609c03eec0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f60a8003980 tx=0x7f60a80092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:15.179 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.494+0000 7f60b0ff9640 1 -- 172.21.15.167:0/147700249 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f60a4039510 con 0x7f609c03cad0 2024-06-18T12:58:15.179 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.499+0000 7f60ba207640 1 -- 172.21.15.167:0/147700249 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f60b4105f80 con 0x7f609c03cad0 2024-06-18T12:58:15.179 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.500+0000 7f60b0ff9640 1 -- 172.21.15.167:0/147700249 <== mgr.14150 v2:172.21.15.167:6800/1145517801 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f60b4105f80 con 0x7f609c03cad0 2024-06-18T12:58:15.179 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.500+0000 7f60ba207640 1 -- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f609c03cad0 msgr2=0x7f609c03eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:15.179 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.500+0000 7f60ba207640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f609c03cad0 0x7f609c03eec0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f60a8003980 tx=0x7f60a80092b0 comp rx=0 tx=0).stop 2024-06-18T12:58:15.179 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.500+0000 7f60ba207640 1 -- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b4105ba0 msgr2=0x7f60b419b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:15.179 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.500+0000 7f60ba207640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b4105ba0 0x7f60b419b110 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f60a40327b0 tx=0x7f60a40327e0 comp rx=0 tx=0).stop 2024-06-18T12:58:15.180 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.500+0000 7f60ba207640 1 -- 172.21.15.167:0/147700249 shutdown_connections 2024-06-18T12:58:15.180 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.500+0000 7f60ba207640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f609c03cad0 0x7f609c03eec0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:15.180 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.500+0000 7f60ba207640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:6800/1873453520,v1:172.21.15.167:6801/1873453520] conn(0x7f609c0391c0 0x7f609c03b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:15.180 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.500+0000 7f60ba207640 1 --2- 172.21.15.167:0/147700249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b4105ba0 0x7f60b419b110 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:15.180 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.500+0000 7f60ba207640 1 -- 172.21.15.167:0/147700249 >> 172.21.15.167:0/147700249 conn(0x7f60b40ff780 msgr2=0x7f60b4100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:15.180 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.500+0000 7f60ba207640 1 -- 172.21.15.167:0/147700249 shutdown_connections 2024-06-18T12:58:15.180 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:14.500+0000 7f60ba207640 1 -- 172.21.15.167:0/147700249 wait complete. 2024-06-18T12:58:15.180 INFO:teuthology.orchestra.run.smithi167.stdout:mgr epoch 9 is available 2024-06-18T12:58:15.180 INFO:teuthology.orchestra.run.smithi167.stdout:Generating a dashboard self-signed certificate... 2024-06-18T12:58:15.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:15 smithi167 ceph-mon[29496]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-18T12:58:15.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:15 smithi167 ceph-mon[29496]: mgrmap e11: y(active, since 1.00566s) 2024-06-18T12:58:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:15 smithi167 ceph-mon[29496]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-18T12:58:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:15 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:16.597 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-18T12:58:16.597 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.653+0000 7f6de1282640 1 Processor -- start 2024-06-18T12:58:16.597 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.653+0000 7f6de1282640 1 -- start start 2024-06-18T12:58:16.597 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.653+0000 7f6de1282640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc105b10 0x7f6ddc107f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:16.598 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.653+0000 7f6de1282640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc0796c0 con 0x7f6ddc105b10 2024-06-18T12:58:16.598 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.653+0000 7f6ddaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc105b10 0x7f6ddc107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:16.598 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.653+0000 7f6ddaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc105b10 0x7f6ddc107f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41766/0 (socket says 172.21.15.167:41766) 2024-06-18T12:58:16.598 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.653+0000 7f6ddaffd640 1 -- 172.21.15.167:0/3683115782 learned_addr learned my addr 172.21.15.167:0/3683115782 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:16.598 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.654+0000 7f6ddaffd640 1 -- 172.21.15.167:0/3683115782 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc108440 con 0x7f6ddc105b10 2024-06-18T12:58:16.598 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.654+0000 7f6ddaffd640 1 --2- 172.21.15.167:0/3683115782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc105b10 0x7f6ddc107f00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009820 tx=0x7f6dcc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17015bcbd85e4638 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:16.598 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.654+0000 7f6dd9ffb640 1 -- 172.21.15.167:0/3683115782 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6dcc037070 con 0x7f6ddc105b10 2024-06-18T12:58:16.598 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.655+0000 7f6dd9ffb640 1 -- 172.21.15.167:0/3683115782 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f6dcc0328c0 con 0x7f6ddc105b10 2024-06-18T12:58:16.598 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.655+0000 7f6dd9ffb640 1 -- 172.21.15.167:0/3683115782 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6dcc030600 con 0x7f6ddc105b10 2024-06-18T12:58:16.598 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.655+0000 7f6de1282640 1 -- 172.21.15.167:0/3683115782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc105b10 msgr2=0x7f6ddc107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:16.598 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.655+0000 7f6de1282640 1 --2- 172.21.15.167:0/3683115782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc105b10 0x7f6ddc107f00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009820 tx=0x7f6dcc02a270 comp rx=0 tx=0).stop 2024-06-18T12:58:16.598 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.655+0000 7f6de1282640 1 -- 172.21.15.167:0/3683115782 shutdown_connections 2024-06-18T12:58:16.599 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.655+0000 7f6de1282640 1 --2- 172.21.15.167:0/3683115782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc105b10 0x7f6ddc107f00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:16.599 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.655+0000 7f6de1282640 1 -- 172.21.15.167:0/3683115782 >> 172.21.15.167:0/3683115782 conn(0x7f6ddc0ff750 msgr2=0x7f6ddc101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:16.599 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.655+0000 7f6de1282640 1 -- 172.21.15.167:0/3683115782 shutdown_connections 2024-06-18T12:58:16.599 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.655+0000 7f6de1282640 1 -- 172.21.15.167:0/3683115782 wait complete. 2024-06-18T12:58:16.599 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.656+0000 7f6de1282640 1 Processor -- start 2024-06-18T12:58:16.599 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.656+0000 7f6de1282640 1 -- start start 2024-06-18T12:58:16.599 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.656+0000 7f6de1282640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc105b10 0x7f6ddc198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:16.599 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.656+0000 7f6de1282640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc109400 con 0x7f6ddc105b10 2024-06-18T12:58:16.599 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.656+0000 7f6ddaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc105b10 0x7f6ddc198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:16.599 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.656+0000 7f6ddaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc105b10 0x7f6ddc198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41768/0 (socket says 172.21.15.167:41768) 2024-06-18T12:58:16.600 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.656+0000 7f6ddaffd640 1 -- 172.21.15.167:0/737913429 learned_addr learned my addr 172.21.15.167:0/737913429 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:16.600 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.657+0000 7f6ddaffd640 1 -- 172.21.15.167:0/737913429 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc199420 con 0x7f6ddc105b10 2024-06-18T12:58:16.600 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.657+0000 7f6ddaffd640 1 --2- 172.21.15.167:0/737913429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc105b10 0x7f6ddc198ee0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0327b0 tx=0x7f6dcc0327e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:16.600 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.657+0000 7f6dc3fff640 1 -- 172.21.15.167:0/737913429 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6dcc037040 con 0x7f6ddc105b10 2024-06-18T12:58:16.600 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.657+0000 7f6de1282640 1 -- 172.21.15.167:0/737913429 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc1996b0 con 0x7f6ddc105b10 2024-06-18T12:58:16.600 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.657+0000 7f6dc3fff640 1 -- 172.21.15.167:0/737913429 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f6dcc040070 con 0x7f6ddc105b10 2024-06-18T12:58:16.600 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.657+0000 7f6dc3fff640 1 -- 172.21.15.167:0/737913429 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6dcc03be50 con 0x7f6ddc105b10 2024-06-18T12:58:16.600 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.657+0000 7f6de1282640 1 -- 172.21.15.167:0/737913429 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc19c3a0 con 0x7f6ddc105b10 2024-06-18T12:58:16.600 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.658+0000 7f6de1282640 1 -- 172.21.15.167:0/737913429 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ddc105ef0 con 0x7f6ddc105b10 2024-06-18T12:58:16.600 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.658+0000 7f6dc3fff640 1 -- 172.21.15.167:0/737913429 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f6dcc030ce0 con 0x7f6ddc105b10 2024-06-18T12:58:16.600 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.659+0000 7f6dc3fff640 1 --2- 172.21.15.167:0/737913429 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f6dc4039030 0x7f6dc403b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:16.600 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.659+0000 7f6dc3fff640 1 -- 172.21.15.167:0/737913429 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6dcc06ce20 con 0x7f6ddc105b10 2024-06-18T12:58:16.601 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.659+0000 7f6dda7fc640 1 --2- 172.21.15.167:0/737913429 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f6dc4039030 0x7f6dc403b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:16.601 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.659+0000 7f6dda7fc640 1 --2- 172.21.15.167:0/737913429 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f6dc4039030 0x7f6dc403b4f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00098c0 tx=0x7f6dd0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:16.601 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.663+0000 7f6dc3fff640 1 -- 172.21.15.167:0/737913429 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f6dcc040220 con 0x7f6ddc105b10 2024-06-18T12:58:16.601 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.853+0000 7f6de1282640 1 -- 172.21.15.167:0/737913429 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f6ddc100900 con 0x7f6dc4039030 2024-06-18T12:58:16.601 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.942+0000 7f6dc3fff640 1 -- 172.21.15.167:0/737913429 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f6ddc100900 con 0x7f6dc4039030 2024-06-18T12:58:16.601 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.944+0000 7f6de1282640 1 -- 172.21.15.167:0/737913429 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f6dc4039030 msgr2=0x7f6dc403b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:16.601 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.944+0000 7f6de1282640 1 --2- 172.21.15.167:0/737913429 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f6dc4039030 0x7f6dc403b4f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00098c0 tx=0x7f6dd0006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:16.601 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.944+0000 7f6de1282640 1 -- 172.21.15.167:0/737913429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc105b10 msgr2=0x7f6ddc198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:16.601 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.944+0000 7f6de1282640 1 --2- 172.21.15.167:0/737913429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc105b10 0x7f6ddc198ee0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0327b0 tx=0x7f6dcc0327e0 comp rx=0 tx=0).stop 2024-06-18T12:58:16.601 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.945+0000 7f6de1282640 1 -- 172.21.15.167:0/737913429 shutdown_connections 2024-06-18T12:58:16.601 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.945+0000 7f6de1282640 1 --2- 172.21.15.167:0/737913429 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f6dc4039030 0x7f6dc403b4f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:16.601 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.945+0000 7f6de1282640 1 --2- 172.21.15.167:0/737913429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc105b10 0x7f6ddc198ee0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:16.601 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.945+0000 7f6de1282640 1 -- 172.21.15.167:0/737913429 >> 172.21.15.167:0/737913429 conn(0x7f6ddc0ff750 msgr2=0x7f6ddc1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:16.602 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.945+0000 7f6de1282640 1 -- 172.21.15.167:0/737913429 shutdown_connections 2024-06-18T12:58:16.602 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:15.945+0000 7f6de1282640 1 -- 172.21.15.167:0/737913429 wait complete. 2024-06-18T12:58:16.602 INFO:teuthology.orchestra.run.smithi167.stdout:Creating initial admin user... 2024-06-18T12:58:16.716 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:16 smithi167 ceph-mon[29496]: [18/Jun/2024:12:58:14] ENGINE Bus STARTING 2024-06-18T12:58:16.716 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:16 smithi167 ceph-mon[29496]: [18/Jun/2024:12:58:14] ENGINE Serving on http://172.21.15.167:8765 2024-06-18T12:58:16.716 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:16 smithi167 ceph-mon[29496]: [18/Jun/2024:12:58:14] ENGINE Serving on https://172.21.15.167:7150 2024-06-18T12:58:16.716 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:16 smithi167 ceph-mon[29496]: [18/Jun/2024:12:58:14] ENGINE Bus STARTED 2024-06-18T12:58:16.716 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:16 smithi167 ceph-mon[29496]: [18/Jun/2024:12:58:14] ENGINE Error in HTTPServer.serve 2024-06-18T12:58:16.716 INFO:journalctl@ceph.mon.a.smithi167.stdout: Traceback (most recent call last): 2024-06-18T12:58:16.716 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-18T12:58:16.716 INFO:journalctl@ceph.mon.a.smithi167.stdout: self._connections.run(self.expiration_interval) 2024-06-18T12:58:16.716 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-18T12:58:16.716 INFO:journalctl@ceph.mon.a.smithi167.stdout: self._run(expiration_interval) 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout: s = self.context.wrap_socket( 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout: return self.sslsocket_class._create( 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout: self.do_handshake() 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout: self._sslobj.do_handshake() 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:16.717 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:16.718 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:16 smithi167 ceph-mon[29496]: mgrmap e12: y(active, since 2s) 2024-06-18T12:58:17.596 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:17 smithi167 ceph-mon[29496]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:58:17.596 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:17 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:18.504 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$bxoNZynqqi0dwE1DNnMrp.ZWJQSMu/EMf4Css17BZEzU7o3pOQH/S", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1718715497, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-18T12:58:18.504 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.972+0000 7f551cbec640 1 Processor -- start 2024-06-18T12:58:18.504 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.972+0000 7f551cbec640 1 -- start start 2024-06-18T12:58:18.504 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.973+0000 7f551cbec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55181050b0 0x7f55181054b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:18.504 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.973+0000 7f551cbec640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5518105a80 con 0x7f55181050b0 2024-06-18T12:58:18.504 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.973+0000 7f551659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55181050b0 0x7f55181054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:18.504 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.973+0000 7f551659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55181050b0 0x7f55181054b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41774/0 (socket says 172.21.15.167:41774) 2024-06-18T12:58:18.504 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.973+0000 7f551659c640 1 -- 172.21.15.167:0/2479376898 learned_addr learned my addr 172.21.15.167:0/2479376898 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:18.504 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.973+0000 7f551659c640 1 -- 172.21.15.167:0/2479376898 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5518106260 con 0x7f55181050b0 2024-06-18T12:58:18.505 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.974+0000 7f551659c640 1 --2- 172.21.15.167:0/2479376898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55181050b0 0x7f55181054b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5500009820 tx=0x7f550002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236fca0c72b574c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:18.505 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.974+0000 7f551559a640 1 -- 172.21.15.167:0/2479376898 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5500037070 con 0x7f55181050b0 2024-06-18T12:58:18.505 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.974+0000 7f551559a640 1 -- 172.21.15.167:0/2479376898 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f55000328c0 con 0x7f55181050b0 2024-06-18T12:58:18.505 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.974+0000 7f551559a640 1 -- 172.21.15.167:0/2479376898 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5500030600 con 0x7f55181050b0 2024-06-18T12:58:18.505 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.974+0000 7f551cbec640 1 -- 172.21.15.167:0/2479376898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55181050b0 msgr2=0x7f55181054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:18.505 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.974+0000 7f551cbec640 1 --2- 172.21.15.167:0/2479376898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55181050b0 0x7f55181054b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5500009820 tx=0x7f550002a270 comp rx=0 tx=0).stop 2024-06-18T12:58:18.505 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.975+0000 7f551cbec640 1 -- 172.21.15.167:0/2479376898 shutdown_connections 2024-06-18T12:58:18.505 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.975+0000 7f551cbec640 1 --2- 172.21.15.167:0/2479376898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55181050b0 0x7f55181054b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:18.505 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.975+0000 7f551cbec640 1 -- 172.21.15.167:0/2479376898 >> 172.21.15.167:0/2479376898 conn(0x7f5518100840 msgr2=0x7f5518102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:18.505 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.975+0000 7f551cbec640 1 -- 172.21.15.167:0/2479376898 shutdown_connections 2024-06-18T12:58:18.505 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.975+0000 7f551cbec640 1 -- 172.21.15.167:0/2479376898 wait complete. 2024-06-18T12:58:18.505 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.975+0000 7f551cbec640 1 Processor -- start 2024-06-18T12:58:18.505 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.975+0000 7f551cbec640 1 -- start start 2024-06-18T12:58:18.506 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.975+0000 7f551cbec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55181050b0 0x7f551807aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:18.506 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.975+0000 7f551cbec640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5518109610 con 0x7f55181050b0 2024-06-18T12:58:18.506 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.976+0000 7f551659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55181050b0 0x7f551807aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:18.506 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.976+0000 7f551659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55181050b0 0x7f551807aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41778/0 (socket says 172.21.15.167:41778) 2024-06-18T12:58:18.506 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.976+0000 7f551659c640 1 -- 172.21.15.167:0/2888368496 learned_addr learned my addr 172.21.15.167:0/2888368496 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:18.506 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.976+0000 7f551659c640 1 -- 172.21.15.167:0/2888368496 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551807b030 con 0x7f55181050b0 2024-06-18T12:58:18.506 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.976+0000 7f551659c640 1 --2- 172.21.15.167:0/2888368496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55181050b0 0x7f551807aaf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f55000097f0 tx=0x7f5500030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:18.506 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.976+0000 7f550f7fe640 1 -- 172.21.15.167:0/2888368496 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5500037040 con 0x7f55181050b0 2024-06-18T12:58:18.506 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.976+0000 7f551cbec640 1 -- 172.21.15.167:0/2888368496 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5518079220 con 0x7f55181050b0 2024-06-18T12:58:18.506 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.976+0000 7f550f7fe640 1 -- 172.21.15.167:0/2888368496 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f5500040070 con 0x7f55181050b0 2024-06-18T12:58:18.506 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.976+0000 7f550f7fe640 1 -- 172.21.15.167:0/2888368496 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f550003bdd0 con 0x7f55181050b0 2024-06-18T12:58:18.506 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.976+0000 7f551cbec640 1 -- 172.21.15.167:0/2888368496 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5518079760 con 0x7f55181050b0 2024-06-18T12:58:18.506 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.977+0000 7f550f7fe640 1 -- 172.21.15.167:0/2888368496 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f5500045460 con 0x7f55181050b0 2024-06-18T12:58:18.507 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.977+0000 7f550f7fe640 1 --2- 172.21.15.167:0/2888368496 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f5504038f70 0x7f550403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:18.507 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.977+0000 7f550f7fe640 1 -- 172.21.15.167:0/2888368496 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f550006cc80 con 0x7f55181050b0 2024-06-18T12:58:18.507 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.977+0000 7f5515d9b640 1 --2- 172.21.15.167:0/2888368496 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f5504038f70 0x7f550403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:18.507 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.977+0000 7f551cbec640 1 -- 172.21.15.167:0/2888368496 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55181054b0 con 0x7f55181050b0 2024-06-18T12:58:18.507 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.978+0000 7f5515d9b640 1 --2- 172.21.15.167:0/2888368496 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f5504038f70 0x7f550403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55080098c0 tx=0x7f5508006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:18.507 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:16.982+0000 7f550f7fe640 1 -- 172.21.15.167:0/2888368496 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f550002e200 con 0x7f55181050b0 2024-06-18T12:58:18.507 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:17.181+0000 7f551cbec640 1 -- 172.21.15.167:0/2888368496 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f5518079bf0 con 0x7f5504038f70 2024-06-18T12:58:18.507 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:17.405+0000 7f550f7fe640 1 -- 172.21.15.167:0/2888368496 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f5518079bf0 con 0x7f5504038f70 2024-06-18T12:58:18.507 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:17.408+0000 7f551cbec640 1 -- 172.21.15.167:0/2888368496 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f5504038f70 msgr2=0x7f550403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:18.507 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:17.408+0000 7f551cbec640 1 --2- 172.21.15.167:0/2888368496 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f5504038f70 0x7f550403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55080098c0 tx=0x7f5508006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:18.507 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:17.408+0000 7f551cbec640 1 -- 172.21.15.167:0/2888368496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55181050b0 msgr2=0x7f551807aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:18.507 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:17.408+0000 7f551cbec640 1 --2- 172.21.15.167:0/2888368496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55181050b0 0x7f551807aaf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f55000097f0 tx=0x7f5500030b30 comp rx=0 tx=0).stop 2024-06-18T12:58:18.507 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:17.408+0000 7f551cbec640 1 -- 172.21.15.167:0/2888368496 shutdown_connections 2024-06-18T12:58:18.508 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:17.408+0000 7f551cbec640 1 --2- 172.21.15.167:0/2888368496 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f5504038f70 0x7f550403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:18.508 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:17.408+0000 7f551cbec640 1 --2- 172.21.15.167:0/2888368496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55181050b0 0x7f551807aaf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:18.508 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:17.408+0000 7f551cbec640 1 -- 172.21.15.167:0/2888368496 >> 172.21.15.167:0/2888368496 conn(0x7f5518100840 msgr2=0x7f55181013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:18.508 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:17.408+0000 7f551cbec640 1 -- 172.21.15.167:0/2888368496 shutdown_connections 2024-06-18T12:58:18.508 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:17.408+0000 7f551cbec640 1 -- 172.21.15.167:0/2888368496 wait complete. 2024-06-18T12:58:18.508 INFO:teuthology.orchestra.run.smithi167.stdout:Fetching dashboard port number... 2024-06-18T12:58:18.626 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:18 smithi167 ceph-mon[29496]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:58:19.744 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3197917979' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-18T12:58:19.751 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stdout 8443 2024-06-18T12:58:19.751 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.945+0000 7fdbe5747640 1 Processor -- start 2024-06-18T12:58:19.751 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.945+0000 7fdbe5747640 1 -- start start 2024-06-18T12:58:19.751 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.945+0000 7fdbe5747640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe01050c0 0x7fdbe01054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:19.751 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.945+0000 7fdbe5747640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe0105a90 con 0x7fdbe01050c0 2024-06-18T12:58:19.751 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.945+0000 7fdbdeffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe01050c0 0x7fdbe01054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:19.751 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.945+0000 7fdbdeffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe01050c0 0x7fdbe01054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41792/0 (socket says 172.21.15.167:41792) 2024-06-18T12:58:19.751 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.945+0000 7fdbdeffd640 1 -- 172.21.15.167:0/1941363877 learned_addr learned my addr 172.21.15.167:0/1941363877 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:19.751 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.946+0000 7fdbdeffd640 1 -- 172.21.15.167:0/1941363877 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe01062c0 con 0x7fdbe01050c0 2024-06-18T12:58:19.751 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.946+0000 7fdbdeffd640 1 --2- 172.21.15.167:0/1941363877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe01050c0 0x7fdbe01054c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009e80 tx=0x7fdbd002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfef85704cbb1f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:19.751 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.946+0000 7fdbddffb640 1 -- 172.21.15.167:0/1941363877 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbd0036070 con 0x7fdbe01050c0 2024-06-18T12:58:19.752 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.946+0000 7fdbddffb640 1 -- 172.21.15.167:0/1941363877 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fdbd0031880 con 0x7fdbe01050c0 2024-06-18T12:58:19.752 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.946+0000 7fdbddffb640 1 -- 172.21.15.167:0/1941363877 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbd00077f0 con 0x7fdbe01050c0 2024-06-18T12:58:19.752 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.947+0000 7fdbe5747640 1 -- 172.21.15.167:0/1941363877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe01050c0 msgr2=0x7fdbe01054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:19.752 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.947+0000 7fdbe5747640 1 --2- 172.21.15.167:0/1941363877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe01050c0 0x7fdbe01054c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009e80 tx=0x7fdbd002a7a0 comp rx=0 tx=0).stop 2024-06-18T12:58:19.752 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.947+0000 7fdbe5747640 1 -- 172.21.15.167:0/1941363877 shutdown_connections 2024-06-18T12:58:19.752 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.947+0000 7fdbe5747640 1 --2- 172.21.15.167:0/1941363877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe01050c0 0x7fdbe01054c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:19.752 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.947+0000 7fdbe5747640 1 -- 172.21.15.167:0/1941363877 >> 172.21.15.167:0/1941363877 conn(0x7fdbe01008b0 msgr2=0x7fdbe0102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:19.752 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.947+0000 7fdbe5747640 1 -- 172.21.15.167:0/1941363877 shutdown_connections 2024-06-18T12:58:19.752 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.947+0000 7fdbe5747640 1 -- 172.21.15.167:0/1941363877 wait complete. 2024-06-18T12:58:19.752 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.948+0000 7fdbe5747640 1 Processor -- start 2024-06-18T12:58:19.752 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.948+0000 7fdbe5747640 1 -- start start 2024-06-18T12:58:19.752 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.948+0000 7fdbe5747640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe01050c0 0x7fdbe0198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:19.752 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.948+0000 7fdbe5747640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe0109670 con 0x7fdbe01050c0 2024-06-18T12:58:19.753 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.948+0000 7fdbdeffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe01050c0 0x7fdbe0198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:19.753 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.948+0000 7fdbdeffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe01050c0 0x7fdbe0198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41802/0 (socket says 172.21.15.167:41802) 2024-06-18T12:58:19.753 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.948+0000 7fdbdeffd640 1 -- 172.21.15.167:0/3197917979 learned_addr learned my addr 172.21.15.167:0/3197917979 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:19.753 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.948+0000 7fdbdeffd640 1 -- 172.21.15.167:0/3197917979 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe0199510 con 0x7fdbe01050c0 2024-06-18T12:58:19.753 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.948+0000 7fdbdeffd640 1 --2- 172.21.15.167:0/3197917979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe01050c0 0x7fdbe0198fd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdbd002acc0 tx=0x7fdbd0031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:19.753 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.949+0000 7fdbc7fff640 1 -- 172.21.15.167:0/3197917979 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbd0041030 con 0x7fdbe01050c0 2024-06-18T12:58:19.753 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.949+0000 7fdbc7fff640 1 -- 172.21.15.167:0/3197917979 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fdbd0007c20 con 0x7fdbe01050c0 2024-06-18T12:58:19.753 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.949+0000 7fdbe5747640 1 -- 172.21.15.167:0/3197917979 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe01997a0 con 0x7fdbe01050c0 2024-06-18T12:58:19.753 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.949+0000 7fdbc7fff640 1 -- 172.21.15.167:0/3197917979 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbd0036040 con 0x7fdbe01050c0 2024-06-18T12:58:19.753 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.949+0000 7fdbe5747640 1 -- 172.21.15.167:0/3197917979 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe019c490 con 0x7fdbe01050c0 2024-06-18T12:58:19.753 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.950+0000 7fdbc7fff640 1 -- 172.21.15.167:0/3197917979 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fdbd0044070 con 0x7fdbe01050c0 2024-06-18T12:58:19.753 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.950+0000 7fdbe5747640 1 -- 172.21.15.167:0/3197917979 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe0105540 con 0x7fdbe01050c0 2024-06-18T12:58:19.753 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.950+0000 7fdbc7fff640 1 --2- 172.21.15.167:0/3197917979 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fdbc8039020 0x7fdbc803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:19.753 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.950+0000 7fdbc7fff640 1 -- 172.21.15.167:0/3197917979 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdbd0042070 con 0x7fdbe01050c0 2024-06-18T12:58:19.754 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.950+0000 7fdbde7fc640 1 --2- 172.21.15.167:0/3197917979 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fdbc8039020 0x7fdbc803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:19.754 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.951+0000 7fdbde7fc640 1 --2- 172.21.15.167:0/3197917979 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fdbc8039020 0x7fdbc803b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40098c0 tx=0x7fdbd4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:19.754 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:18.955+0000 7fdbc7fff640 1 -- 172.21.15.167:0/3197917979 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fdbd004b070 con 0x7fdbe01050c0 2024-06-18T12:58:19.754 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:19.131+0000 7fdbe5747640 1 -- 172.21.15.167:0/3197917979 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fdbe019c920 con 0x7fdbe01050c0 2024-06-18T12:58:19.754 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:19.131+0000 7fdbc7fff640 1 -- 172.21.15.167:0/3197917979 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fdbd003de00 con 0x7fdbe01050c0 2024-06-18T12:58:19.754 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:19.133+0000 7fdbe5747640 1 -- 172.21.15.167:0/3197917979 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fdbc8039020 msgr2=0x7fdbc803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:19.754 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:19.133+0000 7fdbe5747640 1 --2- 172.21.15.167:0/3197917979 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fdbc8039020 0x7fdbc803b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40098c0 tx=0x7fdbd4006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:19.754 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:19.133+0000 7fdbe5747640 1 -- 172.21.15.167:0/3197917979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe01050c0 msgr2=0x7fdbe0198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:19.754 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:19.133+0000 7fdbe5747640 1 --2- 172.21.15.167:0/3197917979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe01050c0 0x7fdbe0198fd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdbd002acc0 tx=0x7fdbd0031630 comp rx=0 tx=0).stop 2024-06-18T12:58:19.754 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:19.134+0000 7fdbe5747640 1 -- 172.21.15.167:0/3197917979 shutdown_connections 2024-06-18T12:58:19.754 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:19.134+0000 7fdbe5747640 1 --2- 172.21.15.167:0/3197917979 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fdbc8039020 0x7fdbc803b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:19.754 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:19.134+0000 7fdbe5747640 1 --2- 172.21.15.167:0/3197917979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe01050c0 0x7fdbe0198fd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:19.755 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:19.134+0000 7fdbe5747640 1 -- 172.21.15.167:0/3197917979 >> 172.21.15.167:0/3197917979 conn(0x7fdbe01008b0 msgr2=0x7fdbe01013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:19.755 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:19.134+0000 7fdbe5747640 1 -- 172.21.15.167:0/3197917979 shutdown_connections 2024-06-18T12:58:19.755 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:19.134+0000 7fdbe5747640 1 -- 172.21.15.167:0/3197917979 wait complete. 2024-06-18T12:58:19.758 INFO:teuthology.orchestra.run.smithi167.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-18T12:58:19.758 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stdout disabled 2024-06-18T12:58:19.766 INFO:teuthology.orchestra.run.smithi167.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-18T12:58:19.766 INFO:teuthology.orchestra.run.smithi167.stdout:systemctl: stdout inactive 2024-06-18T12:58:19.766 INFO:teuthology.orchestra.run.smithi167.stdout:firewalld.service is not enabled 2024-06-18T12:58:19.766 INFO:teuthology.orchestra.run.smithi167.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-18T12:58:19.768 INFO:teuthology.orchestra.run.smithi167.stdout:Ceph Dashboard is now available at: 2024-06-18T12:58:19.768 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:58:19.768 INFO:teuthology.orchestra.run.smithi167.stdout: URL: https://smithi167.front.sepia.ceph.com:8443/ 2024-06-18T12:58:19.768 INFO:teuthology.orchestra.run.smithi167.stdout: User: admin 2024-06-18T12:58:19.768 INFO:teuthology.orchestra.run.smithi167.stdout: Password: t592wxvsh5 2024-06-18T12:58:19.768 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:58:19.769 INFO:teuthology.orchestra.run.smithi167.stdout:Saving cluster configuration to /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config directory 2024-06-18T12:58:19.769 INFO:teuthology.orchestra.run.smithi167.stdout:Enabling autotune for osd_memory_target 2024-06-18T12:58:20.642 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:20 smithi167 ceph-mon[29496]: mgrmap e13: y(active, since 6s) 2024-06-18T12:58:20.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.162+0000 7f0ebce73640 1 Processor -- start 2024-06-18T12:58:20.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.162+0000 7f0ebce73640 1 -- start start 2024-06-18T12:58:20.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.162+0000 7f0ebce73640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eb81050c0 0x7f0eb81054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:20.883 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.162+0000 7f0ebce73640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8105a90 con 0x7f0eb81050c0 2024-06-18T12:58:20.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.162+0000 7f0eb659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eb81050c0 0x7f0eb81054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:20.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.162+0000 7f0eb659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eb81050c0 0x7f0eb81054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41810/0 (socket says 172.21.15.167:41810) 2024-06-18T12:58:20.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.162+0000 7f0eb659c640 1 -- 172.21.15.167:0/798596225 learned_addr learned my addr 172.21.15.167:0/798596225 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:20.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.163+0000 7f0eb659c640 1 -- 172.21.15.167:0/798596225 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb81062c0 con 0x7f0eb81050c0 2024-06-18T12:58:20.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.163+0000 7f0eb659c640 1 --2- 172.21.15.167:0/798596225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eb81050c0 0x7f0eb81054c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009820 tx=0x7f0ea802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b958f26c1ae534 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:20.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.163+0000 7f0eb559a640 1 -- 172.21.15.167:0/798596225 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ea8037070 con 0x7f0eb81050c0 2024-06-18T12:58:20.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.164+0000 7f0eb559a640 1 -- 172.21.15.167:0/798596225 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f0ea80328c0 con 0x7f0eb81050c0 2024-06-18T12:58:20.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.164+0000 7f0eb559a640 1 -- 172.21.15.167:0/798596225 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ea8030600 con 0x7f0eb81050c0 2024-06-18T12:58:20.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.164+0000 7f0ebce73640 1 -- 172.21.15.167:0/798596225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eb81050c0 msgr2=0x7f0eb81054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:20.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.164+0000 7f0ebce73640 1 --2- 172.21.15.167:0/798596225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eb81050c0 0x7f0eb81054c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009820 tx=0x7f0ea802a270 comp rx=0 tx=0).stop 2024-06-18T12:58:20.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.164+0000 7f0ebce73640 1 -- 172.21.15.167:0/798596225 shutdown_connections 2024-06-18T12:58:20.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.164+0000 7f0ebce73640 1 --2- 172.21.15.167:0/798596225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eb81050c0 0x7f0eb81054c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:20.884 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.164+0000 7f0ebce73640 1 -- 172.21.15.167:0/798596225 >> 172.21.15.167:0/798596225 conn(0x7f0eb81008b0 msgr2=0x7f0eb8102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:20.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.164+0000 7f0ebce73640 1 -- 172.21.15.167:0/798596225 shutdown_connections 2024-06-18T12:58:20.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.164+0000 7f0ebce73640 1 -- 172.21.15.167:0/798596225 wait complete. 2024-06-18T12:58:20.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.165+0000 7f0ebce73640 1 Processor -- start 2024-06-18T12:58:20.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.165+0000 7f0ebce73640 1 -- start start 2024-06-18T12:58:20.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.165+0000 7f0ebce73640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eb81050c0 0x7f0eb8198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:20.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.165+0000 7f0ebce73640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8109670 con 0x7f0eb81050c0 2024-06-18T12:58:20.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.165+0000 7f0eb659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eb81050c0 0x7f0eb8198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:20.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.165+0000 7f0eb659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eb81050c0 0x7f0eb8198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41820/0 (socket says 172.21.15.167:41820) 2024-06-18T12:58:20.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.165+0000 7f0eb659c640 1 -- 172.21.15.167:0/775736210 learned_addr learned my addr 172.21.15.167:0/775736210 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:20.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.165+0000 7f0eb659c640 1 -- 172.21.15.167:0/775736210 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb8199470 con 0x7f0eb81050c0 2024-06-18T12:58:20.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.166+0000 7f0eb659c640 1 --2- 172.21.15.167:0/775736210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eb81050c0 0x7f0eb8198f30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009940 tx=0x7f0ea80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:20.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.166+0000 7f0ea77fe640 1 -- 172.21.15.167:0/775736210 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ea8037040 con 0x7f0eb81050c0 2024-06-18T12:58:20.885 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.166+0000 7f0ea77fe640 1 -- 172.21.15.167:0/775736210 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f0ea8040070 con 0x7f0eb81050c0 2024-06-18T12:58:20.886 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.166+0000 7f0ebce73640 1 -- 172.21.15.167:0/775736210 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb8199700 con 0x7f0eb81050c0 2024-06-18T12:58:20.886 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.166+0000 7f0ea77fe640 1 -- 172.21.15.167:0/775736210 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ea803bd50 con 0x7f0eb81050c0 2024-06-18T12:58:20.886 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.166+0000 7f0ebce73640 1 -- 172.21.15.167:0/775736210 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb819c3f0 con 0x7f0eb81050c0 2024-06-18T12:58:20.886 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.167+0000 7f0ea77fe640 1 -- 172.21.15.167:0/775736210 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f0ea8045460 con 0x7f0eb81050c0 2024-06-18T12:58:20.886 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.167+0000 7f0ebce73640 1 -- 172.21.15.167:0/775736210 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb81054c0 con 0x7f0eb81050c0 2024-06-18T12:58:20.886 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.167+0000 7f0ea77fe640 1 --2- 172.21.15.167:0/775736210 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f0ea00390c0 0x7f0ea003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:20.886 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.167+0000 7f0ea77fe640 1 -- 172.21.15.167:0/775736210 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0ea806c670 con 0x7f0eb81050c0 2024-06-18T12:58:20.886 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.167+0000 7f0eb5d9b640 1 --2- 172.21.15.167:0/775736210 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f0ea00390c0 0x7f0ea003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:20.886 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.167+0000 7f0eb5d9b640 1 --2- 172.21.15.167:0/775736210 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f0ea00390c0 0x7f0ea003b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0098c0 tx=0x7f0eac006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:20.886 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.172+0000 7f0ea77fe640 1 -- 172.21.15.167:0/775736210 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f0ea8043360 con 0x7f0eb81050c0 2024-06-18T12:58:20.886 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.345+0000 7f0ebce73640 1 -- 172.21.15.167:0/775736210 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f0eb8079b90 con 0x7f0eb81050c0 2024-06-18T12:58:20.886 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.345+0000 7f0ea77fe640 1 -- 172.21.15.167:0/775736210 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f0ea8042df0 con 0x7f0eb81050c0 2024-06-18T12:58:20.887 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.347+0000 7f0ebce73640 1 -- 172.21.15.167:0/775736210 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f0ea00390c0 msgr2=0x7f0ea003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:20.887 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.347+0000 7f0ebce73640 1 --2- 172.21.15.167:0/775736210 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f0ea00390c0 0x7f0ea003b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0098c0 tx=0x7f0eac006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:20.887 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.347+0000 7f0ebce73640 1 -- 172.21.15.167:0/775736210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eb81050c0 msgr2=0x7f0eb8198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:20.887 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.347+0000 7f0ebce73640 1 --2- 172.21.15.167:0/775736210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eb81050c0 0x7f0eb8198f30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009940 tx=0x7f0ea80327b0 comp rx=0 tx=0).stop 2024-06-18T12:58:20.887 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.347+0000 7f0ebce73640 1 -- 172.21.15.167:0/775736210 shutdown_connections 2024-06-18T12:58:20.887 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.347+0000 7f0ebce73640 1 --2- 172.21.15.167:0/775736210 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f0ea00390c0 0x7f0ea003b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:20.887 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.347+0000 7f0ebce73640 1 --2- 172.21.15.167:0/775736210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eb81050c0 0x7f0eb8198f30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:20.887 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.347+0000 7f0ebce73640 1 -- 172.21.15.167:0/775736210 >> 172.21.15.167:0/775736210 conn(0x7f0eb81008b0 msgr2=0x7f0eb8101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:20.887 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.348+0000 7f0ebce73640 1 -- 172.21.15.167:0/775736210 shutdown_connections 2024-06-18T12:58:20.887 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:20.348+0000 7f0ebce73640 1 -- 172.21.15.167:0/775736210 wait complete. 2024-06-18T12:58:22.074 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.225+0000 7fa4d8d92640 1 Processor -- start 2024-06-18T12:58:22.075 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.225+0000 7fa4d8d92640 1 -- start start 2024-06-18T12:58:22.075 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.225+0000 7fa4d8d92640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4d407b690 0x7fa4d4079af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:22.075 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.225+0000 7fa4d8d92640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d407a0c0 con 0x7fa4d407b690 2024-06-18T12:58:22.075 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.225+0000 7fa4d259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4d407b690 0x7fa4d4079af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:22.075 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.225+0000 7fa4d259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4d407b690 0x7fa4d4079af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41826/0 (socket says 172.21.15.167:41826) 2024-06-18T12:58:22.075 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.225+0000 7fa4d259c640 1 -- 172.21.15.167:0/1203643912 learned_addr learned my addr 172.21.15.167:0/1203643912 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:22.075 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.226+0000 7fa4d259c640 1 -- 172.21.15.167:0/1203643912 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d407a240 con 0x7fa4d407b690 2024-06-18T12:58:22.075 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.226+0000 7fa4d259c640 1 --2- 172.21.15.167:0/1203643912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4d407b690 0x7fa4d4079af0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009820 tx=0x7fa4c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a5253ab7cb027f server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:22.075 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.226+0000 7fa4d159a640 1 -- 172.21.15.167:0/1203643912 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4c8037070 con 0x7fa4d407b690 2024-06-18T12:58:22.075 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.227+0000 7fa4d159a640 1 -- 172.21.15.167:0/1203643912 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fa4c80328c0 con 0x7fa4d407b690 2024-06-18T12:58:22.075 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.227+0000 7fa4d159a640 1 -- 172.21.15.167:0/1203643912 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4c8030600 con 0x7fa4d407b690 2024-06-18T12:58:22.076 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.227+0000 7fa4d8d92640 1 -- 172.21.15.167:0/1203643912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4d407b690 msgr2=0x7fa4d4079af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:22.076 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.227+0000 7fa4d8d92640 1 --2- 172.21.15.167:0/1203643912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4d407b690 0x7fa4d4079af0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009820 tx=0x7fa4c802a270 comp rx=0 tx=0).stop 2024-06-18T12:58:22.076 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.227+0000 7fa4d8d92640 1 -- 172.21.15.167:0/1203643912 shutdown_connections 2024-06-18T12:58:22.076 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.227+0000 7fa4d8d92640 1 --2- 172.21.15.167:0/1203643912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4d407b690 0x7fa4d4079af0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:22.076 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.227+0000 7fa4d8d92640 1 -- 172.21.15.167:0/1203643912 >> 172.21.15.167:0/1203643912 conn(0x7fa4d41008b0 msgr2=0x7fa4d4102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:22.076 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.227+0000 7fa4d8d92640 1 -- 172.21.15.167:0/1203643912 shutdown_connections 2024-06-18T12:58:22.076 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.227+0000 7fa4d8d92640 1 -- 172.21.15.167:0/1203643912 wait complete. 2024-06-18T12:58:22.076 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.228+0000 7fa4d8d92640 1 Processor -- start 2024-06-18T12:58:22.076 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.228+0000 7fa4d8d92640 1 -- start start 2024-06-18T12:58:22.076 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.228+0000 7fa4d8d92640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4d407b690 0x7fa4d419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:22.076 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.228+0000 7fa4d8d92640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d410b630 con 0x7fa4d407b690 2024-06-18T12:58:22.076 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.228+0000 7fa4d259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4d407b690 0x7fa4d419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:22.076 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.228+0000 7fa4d259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4d407b690 0x7fa4d419b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41830/0 (socket says 172.21.15.167:41830) 2024-06-18T12:58:22.077 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.228+0000 7fa4d259c640 1 -- 172.21.15.167:0/3213803223 learned_addr learned my addr 172.21.15.167:0/3213803223 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:22.077 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.229+0000 7fa4d259c640 1 -- 172.21.15.167:0/3213803223 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d419b650 con 0x7fa4d407b690 2024-06-18T12:58:22.077 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.229+0000 7fa4d259c640 1 --2- 172.21.15.167:0/3213803223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4d407b690 0x7fa4d419b110 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa4c80327b0 tx=0x7fa4c80327e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:22.077 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.229+0000 7fa4c37fe640 1 -- 172.21.15.167:0/3213803223 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4c8037040 con 0x7fa4d407b690 2024-06-18T12:58:22.077 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.229+0000 7fa4c37fe640 1 -- 172.21.15.167:0/3213803223 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fa4c8040070 con 0x7fa4d407b690 2024-06-18T12:58:22.077 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.229+0000 7fa4d8d92640 1 -- 172.21.15.167:0/3213803223 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d419b8e0 con 0x7fa4d407b690 2024-06-18T12:58:22.077 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.229+0000 7fa4c37fe640 1 -- 172.21.15.167:0/3213803223 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4c803be50 con 0x7fa4d407b690 2024-06-18T12:58:22.077 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.229+0000 7fa4d8d92640 1 -- 172.21.15.167:0/3213803223 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d419e5d0 con 0x7fa4d407b690 2024-06-18T12:58:22.077 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.230+0000 7fa4c37fe640 1 -- 172.21.15.167:0/3213803223 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fa4c8045460 con 0x7fa4d407b690 2024-06-18T12:58:22.077 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.230+0000 7fa4c37fe640 1 --2- 172.21.15.167:0/3213803223 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fa4bc039200 0x7fa4bc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:22.077 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.230+0000 7fa4c37fe640 1 -- 172.21.15.167:0/3213803223 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa4c806c5f0 con 0x7fa4d407b690 2024-06-18T12:58:22.077 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.230+0000 7fa4d1d9b640 1 --2- 172.21.15.167:0/3213803223 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fa4bc039200 0x7fa4bc03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:22.077 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.230+0000 7fa4d8d92640 1 -- 172.21.15.167:0/3213803223 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d4079af0 con 0x7fa4d407b690 2024-06-18T12:58:22.078 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.231+0000 7fa4d1d9b640 1 --2- 172.21.15.167:0/3213803223 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fa4bc039200 0x7fa4bc03b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40098c0 tx=0x7fa4c4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:22.078 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.235+0000 7fa4c37fe640 1 -- 172.21.15.167:0/3213803223 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fa4c8039550 con 0x7fa4d407b690 2024-06-18T12:58:22.078 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.502+0000 7fa4d8d92640 1 -- 172.21.15.167:0/3213803223 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fa4d419e8b0 con 0x7fa4d407b690 2024-06-18T12:58:22.078 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.505+0000 7fa4c37fe640 1 -- 172.21.15.167:0/3213803223 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fa4c80392f0 con 0x7fa4d407b690 2024-06-18T12:58:22.078 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-18T12:58:22.078 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.507+0000 7fa4d8d92640 1 -- 172.21.15.167:0/3213803223 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fa4bc039200 msgr2=0x7fa4bc03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:22.078 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.507+0000 7fa4d8d92640 1 --2- 172.21.15.167:0/3213803223 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fa4bc039200 0x7fa4bc03b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40098c0 tx=0x7fa4c4006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:22.078 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.507+0000 7fa4d8d92640 1 -- 172.21.15.167:0/3213803223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4d407b690 msgr2=0x7fa4d419b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:22.078 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.507+0000 7fa4d8d92640 1 --2- 172.21.15.167:0/3213803223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4d407b690 0x7fa4d419b110 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa4c80327b0 tx=0x7fa4c80327e0 comp rx=0 tx=0).stop 2024-06-18T12:58:22.078 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.507+0000 7fa4d8d92640 1 -- 172.21.15.167:0/3213803223 shutdown_connections 2024-06-18T12:58:22.078 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.507+0000 7fa4d8d92640 1 --2- 172.21.15.167:0/3213803223 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fa4bc039200 0x7fa4bc03b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:22.078 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.507+0000 7fa4d8d92640 1 --2- 172.21.15.167:0/3213803223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4d407b690 0x7fa4d419b110 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:22.078 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.507+0000 7fa4d8d92640 1 -- 172.21.15.167:0/3213803223 >> 172.21.15.167:0/3213803223 conn(0x7fa4d41008b0 msgr2=0x7fa4d4101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:22.079 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.507+0000 7fa4d8d92640 1 -- 172.21.15.167:0/3213803223 shutdown_connections 2024-06-18T12:58:22.079 INFO:teuthology.orchestra.run.smithi167.stdout:/usr/bin/ceph: stderr 2024-06-18T12:58:21.507+0000 7fa4d8d92640 1 -- 172.21.15.167:0/3213803223 wait complete. 2024-06-18T12:58:22.079 INFO:teuthology.orchestra.run.smithi167.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-18T12:58:22.079 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:58:22.079 INFO:teuthology.orchestra.run.smithi167.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-18T12:58:22.079 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:58:22.079 INFO:teuthology.orchestra.run.smithi167.stdout:Or, if you are only running a single cluster on this host: 2024-06-18T12:58:22.079 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:58:22.079 INFO:teuthology.orchestra.run.smithi167.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-18T12:58:22.079 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:58:22.079 INFO:teuthology.orchestra.run.smithi167.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-18T12:58:22.079 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:58:22.079 INFO:teuthology.orchestra.run.smithi167.stdout: ceph telemetry on 2024-06-18T12:58:22.079 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:58:22.080 INFO:teuthology.orchestra.run.smithi167.stdout:For more information see: 2024-06-18T12:58:22.080 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:58:22.080 INFO:teuthology.orchestra.run.smithi167.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-18T12:58:22.080 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:58:22.080 INFO:teuthology.orchestra.run.smithi167.stdout:Bootstrap complete. 2024-06-18T12:58:22.111 INFO:tasks.cephadm:Fetching config... 2024-06-18T12:58:22.111 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T12:58:22.111 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-18T12:58:22.127 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-18T12:58:22.127 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T12:58:22.127 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-18T12:58:22.184 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-18T12:58:22.184 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T12:58:22.184 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-18T12:58:22.249 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-18T12:58:22.249 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T12:58:22.249 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-18T12:58:22.305 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-18T12:58:22.306 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/0udb4frrnW2NbdrCUUBbKqccVq5E0FpRcuDdYbtYNAVJ0xQPC8KgRmPYY1C3HVsi0YKYwtvsI0R99zRw5oaSv/XH/6zXlJpU9mCGu8av/nwUl3Dr5P2QmxSreGvsMT/RcV+hdip9BL75fxgEGgrgYBnFkrIo1eE7z5YXhajTU1vGRhp/nEe993cWJNRiGyQpTG6L+L7MUe3pPQ4EJUWkIe110ghu+3nj9sz3RWkBR1UbUGjZXCZ8kQCFGkRhvzanexgsKBuWVonCMb/+D10dl8KIKRI5ZnVQ9shzS9qM3KSZtpZexLevvclo7eIVAJhEaItpKlMZPsL0QK0IqnVO6euOCRMAxjEZdWZJttdYBlX85tTh67Ljs7m7UW9nuDSdF6SfS/7JwWXIpyg4gaseVHeVG5u0K8nXfJCNbVUOf8GEM7aKbXmWVzBqNHAd21gzAPVfZn7NoX5471D8bKnMk+AjZQIA6bDHWs3ct1JLZz/2U9T+qL5sRoBpDmWB+Vs= ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-18T12:58:22.380 INFO:teuthology.orchestra.run.smithi167.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/0udb4frrnW2NbdrCUUBbKqccVq5E0FpRcuDdYbtYNAVJ0xQPC8KgRmPYY1C3HVsi0YKYwtvsI0R99zRw5oaSv/XH/6zXlJpU9mCGu8av/nwUl3Dr5P2QmxSreGvsMT/RcV+hdip9BL75fxgEGgrgYBnFkrIo1eE7z5YXhajTU1vGRhp/nEe993cWJNRiGyQpTG6L+L7MUe3pPQ4EJUWkIe110ghu+3nj9sz3RWkBR1UbUGjZXCZ8kQCFGkRhvzanexgsKBuWVonCMb/+D10dl8KIKRI5ZnVQ9shzS9qM3KSZtpZexLevvclo7eIVAJhEaItpKlMZPsL0QK0IqnVO6euOCRMAxjEZdWZJttdYBlX85tTh67Ljs7m7UW9nuDSdF6SfS/7JwWXIpyg4gaseVHeVG5u0K8nXfJCNbVUOf8GEM7aKbXmWVzBqNHAd21gzAPVfZn7NoX5471D8bKnMk+AjZQIA6bDHWs3ct1JLZz/2U9T+qL5sRoBpDmWB+Vs= ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:58:22.391 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/0udb4frrnW2NbdrCUUBbKqccVq5E0FpRcuDdYbtYNAVJ0xQPC8KgRmPYY1C3HVsi0YKYwtvsI0R99zRw5oaSv/XH/6zXlJpU9mCGu8av/nwUl3Dr5P2QmxSreGvsMT/RcV+hdip9BL75fxgEGgrgYBnFkrIo1eE7z5YXhajTU1vGRhp/nEe993cWJNRiGyQpTG6L+L7MUe3pPQ4EJUWkIe110ghu+3nj9sz3RWkBR1UbUGjZXCZ8kQCFGkRhvzanexgsKBuWVonCMb/+D10dl8KIKRI5ZnVQ9shzS9qM3KSZtpZexLevvclo7eIVAJhEaItpKlMZPsL0QK0IqnVO6euOCRMAxjEZdWZJttdYBlX85tTh67Ljs7m7UW9nuDSdF6SfS/7JwWXIpyg4gaseVHeVG5u0K8nXfJCNbVUOf8GEM7aKbXmWVzBqNHAd21gzAPVfZn7NoX5471D8bKnMk+AjZQIA6bDHWs3ct1JLZz/2U9T+qL5sRoBpDmWB+Vs= ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-18T12:58:22.432 INFO:teuthology.orchestra.run.smithi179.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/0udb4frrnW2NbdrCUUBbKqccVq5E0FpRcuDdYbtYNAVJ0xQPC8KgRmPYY1C3HVsi0YKYwtvsI0R99zRw5oaSv/XH/6zXlJpU9mCGu8av/nwUl3Dr5P2QmxSreGvsMT/RcV+hdip9BL75fxgEGgrgYBnFkrIo1eE7z5YXhajTU1vGRhp/nEe993cWJNRiGyQpTG6L+L7MUe3pPQ4EJUWkIe110ghu+3nj9sz3RWkBR1UbUGjZXCZ8kQCFGkRhvzanexgsKBuWVonCMb/+D10dl8KIKRI5ZnVQ9shzS9qM3KSZtpZexLevvclo7eIVAJhEaItpKlMZPsL0QK0IqnVO6euOCRMAxjEZdWZJttdYBlX85tTh67Ljs7m7UW9nuDSdF6SfS/7JwWXIpyg4gaseVHeVG5u0K8nXfJCNbVUOf8GEM7aKbXmWVzBqNHAd21gzAPVfZn7NoX5471D8bKnMk+AjZQIA6bDHWs3ct1JLZz/2U9T+qL5sRoBpDmWB+Vs= ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:58:22.445 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-18T12:58:22.595 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3213803223' entity='client.admin' 2024-06-18T12:58:22.696 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T12:58:24.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.718+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1729388111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f340ffcd0 msgr2=0x7f1f341000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:24.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.718+0000 7f1f3a00c640 1 --2- 172.21.15.167:0/1729388111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f340ffcd0 0x7f1f341000d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009880 tx=0x7f1f2402a5d0 comp rx=0 tx=0).stop 2024-06-18T12:58:24.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.718+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1729388111 shutdown_connections 2024-06-18T12:58:24.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.718+0000 7f1f3a00c640 1 --2- 172.21.15.167:0/1729388111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f340ffcd0 0x7f1f341000d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:24.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.718+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1729388111 >> 172.21.15.167:0/1729388111 conn(0x7f1f340fb460 msgr2=0x7f1f340fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:24.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.718+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1729388111 shutdown_connections 2024-06-18T12:58:24.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.718+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1729388111 wait complete. 2024-06-18T12:58:24.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.719+0000 7f1f3a00c640 1 Processor -- start 2024-06-18T12:58:24.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.719+0000 7f1f3a00c640 1 -- start start 2024-06-18T12:58:24.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.719+0000 7f1f3a00c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f340ffcd0 0x7f1f34195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:24.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.719+0000 7f1f3a00c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f34105260 con 0x7f1f340ffcd0 2024-06-18T12:58:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.719+0000 7f1f337fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f340ffcd0 0x7f1f34195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.719+0000 7f1f337fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f340ffcd0 0x7f1f34195b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:57664/0 (socket says 172.21.15.167:57664) 2024-06-18T12:58:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.719+0000 7f1f337fe640 1 -- 172.21.15.167:0/1027216719 learned_addr learned my addr 172.21.15.167:0/1027216719 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.719+0000 7f1f337fe640 1 -- 172.21.15.167:0/1027216719 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f34196040 con 0x7f1f340ffcd0 2024-06-18T12:58:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.720+0000 7f1f337fe640 1 --2- 172.21.15.167:0/1027216719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f340ffcd0 0x7f1f34195b00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f240057c0 tx=0x7f1f24005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.720+0000 7f1f30ff9640 1 -- 172.21.15.167:0/1027216719 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f2403f070 con 0x7f1f340ffcd0 2024-06-18T12:58:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.720+0000 7f1f30ff9640 1 -- 172.21.15.167:0/1027216719 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f1f24003ba0 con 0x7f1f340ffcd0 2024-06-18T12:58:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.720+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1027216719 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f341962d0 con 0x7f1f340ffcd0 2024-06-18T12:58:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.720+0000 7f1f30ff9640 1 -- 172.21.15.167:0/1027216719 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f24035030 con 0x7f1f340ffcd0 2024-06-18T12:58:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.720+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1027216719 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f34198fc0 con 0x7f1f340ffcd0 2024-06-18T12:58:24.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.721+0000 7f1f30ff9640 1 -- 172.21.15.167:0/1027216719 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1f24042070 con 0x7f1f340ffcd0 2024-06-18T12:58:24.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.721+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1027216719 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f341059e0 con 0x7f1f340ffcd0 2024-06-18T12:58:24.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.721+0000 7f1f30ff9640 1 --2- 172.21.15.167:0/1027216719 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f1f1c039060 0x7f1f1c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:24.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.721+0000 7f1f30ff9640 1 -- 172.21.15.167:0/1027216719 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1f2406cf60 con 0x7f1f340ffcd0 2024-06-18T12:58:24.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.721+0000 7f1f32ffd640 1 --2- 172.21.15.167:0/1027216719 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f1f1c039060 0x7f1f1c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:24.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.721+0000 7f1f32ffd640 1 --2- 172.21.15.167:0/1027216719 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f1f1c039060 0x7f1f1c03b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1f280098c0 tx=0x7f1f28006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:24.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.726+0000 7f1f30ff9640 1 -- 172.21.15.167:0/1027216719 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f1f240433b0 con 0x7f1f340ffcd0 2024-06-18T12:58:24.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.899+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1027216719 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f1f3418f050 con 0x7f1f340ffcd0 2024-06-18T12:58:24.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.901+0000 7f1f30ff9640 1 -- 172.21.15.167:0/1027216719 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f1f240307a0 con 0x7f1f340ffcd0 2024-06-18T12:58:24.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.904+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1027216719 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f1f1c039060 msgr2=0x7f1f1c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:24.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.904+0000 7f1f3a00c640 1 --2- 172.21.15.167:0/1027216719 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f1f1c039060 0x7f1f1c03b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1f280098c0 tx=0x7f1f28006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:24.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.904+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1027216719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f340ffcd0 msgr2=0x7f1f34195b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:24.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.904+0000 7f1f3a00c640 1 --2- 172.21.15.167:0/1027216719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f340ffcd0 0x7f1f34195b00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f240057c0 tx=0x7f1f24005160 comp rx=0 tx=0).stop 2024-06-18T12:58:24.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.904+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1027216719 shutdown_connections 2024-06-18T12:58:24.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.904+0000 7f1f3a00c640 1 --2- 172.21.15.167:0/1027216719 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f1f1c039060 0x7f1f1c03b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:24.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.904+0000 7f1f3a00c640 1 --2- 172.21.15.167:0/1027216719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f340ffcd0 0x7f1f34195b00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:24.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.904+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1027216719 >> 172.21.15.167:0/1027216719 conn(0x7f1f340fb460 msgr2=0x7f1f340698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:24.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.904+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1027216719 shutdown_connections 2024-06-18T12:58:24.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:24.904+0000 7f1f3a00c640 1 -- 172.21.15.167:0/1027216719 wait complete. 2024-06-18T12:58:25.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:24 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:25.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:24 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:25.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:24 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-06-18T12:58:25.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:24 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:25.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:24 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:58:25.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:24 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:25.491 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-18T12:58:25.491 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-18T12:58:25.727 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T12:58:26.108 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1027216719' entity='client.admin' 2024-06-18T12:58:26.108 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:25 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:58:26.108 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:25 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:27.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.141+0000 7fa69a9df640 1 -- 172.21.15.167:0/1931182451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694100dd0 msgr2=0x7fa6941011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:27.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.141+0000 7fa69a9df640 1 --2- 172.21.15.167:0/1931182451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694100dd0 0x7fa6941011d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa684009880 tx=0x7fa68402a5d0 comp rx=0 tx=0).stop 2024-06-18T12:58:27.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.141+0000 7fa69a9df640 1 -- 172.21.15.167:0/1931182451 shutdown_connections 2024-06-18T12:58:27.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.141+0000 7fa69a9df640 1 --2- 172.21.15.167:0/1931182451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694100dd0 0x7fa6941011d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:27.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.141+0000 7fa69a9df640 1 -- 172.21.15.167:0/1931182451 >> 172.21.15.167:0/1931182451 conn(0x7fa6940fc5a0 msgr2=0x7fa6940fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:27.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.141+0000 7fa69a9df640 1 -- 172.21.15.167:0/1931182451 shutdown_connections 2024-06-18T12:58:27.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.141+0000 7fa69a9df640 1 -- 172.21.15.167:0/1931182451 wait complete. 2024-06-18T12:58:27.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.142+0000 7fa69a9df640 1 Processor -- start 2024-06-18T12:58:27.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.142+0000 7fa69a9df640 1 -- start start 2024-06-18T12:58:27.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.142+0000 7fa69a9df640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694100dd0 0x7fa694075660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:27.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.142+0000 7fa69a9df640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa694106360 con 0x7fa694100dd0 2024-06-18T12:58:27.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.142+0000 7fa693fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694100dd0 0x7fa694075660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:27.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.142+0000 7fa693fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694100dd0 0x7fa694075660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:57678/0 (socket says 172.21.15.167:57678) 2024-06-18T12:58:27.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.142+0000 7fa693fff640 1 -- 172.21.15.167:0/1179388886 learned_addr learned my addr 172.21.15.167:0/1179388886 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:27.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.142+0000 7fa693fff640 1 -- 172.21.15.167:0/1179388886 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa694075ba0 con 0x7fa694100dd0 2024-06-18T12:58:27.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.143+0000 7fa693fff640 1 --2- 172.21.15.167:0/1179388886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694100dd0 0x7fa694075660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa6840057c0 tx=0x7fa684005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:27.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.143+0000 7fa6917fa640 1 -- 172.21.15.167:0/1179388886 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa68403f070 con 0x7fa694100dd0 2024-06-18T12:58:27.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.143+0000 7fa6917fa640 1 -- 172.21.15.167:0/1179388886 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fa684003ba0 con 0x7fa694100dd0 2024-06-18T12:58:27.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.143+0000 7fa69a9df640 1 -- 172.21.15.167:0/1179388886 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa694073d90 con 0x7fa694100dd0 2024-06-18T12:58:27.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.143+0000 7fa6917fa640 1 -- 172.21.15.167:0/1179388886 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa684035030 con 0x7fa694100dd0 2024-06-18T12:58:27.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.143+0000 7fa69a9df640 1 -- 172.21.15.167:0/1179388886 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6940742d0 con 0x7fa694100dd0 2024-06-18T12:58:27.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.144+0000 7fa6917fa640 1 -- 172.21.15.167:0/1179388886 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fa684042070 con 0x7fa694100dd0 2024-06-18T12:58:27.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.144+0000 7fa6917fa640 1 --2- 172.21.15.167:0/1179388886 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fa67c039060 0x7fa67c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:27.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.144+0000 7fa6917fa640 1 -- 172.21.15.167:0/1179388886 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa68406cdf0 con 0x7fa694100dd0 2024-06-18T12:58:27.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.144+0000 7fa69a9df640 1 -- 172.21.15.167:0/1179388886 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa694106ae0 con 0x7fa694100dd0 2024-06-18T12:58:27.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.144+0000 7fa6937fe640 1 --2- 172.21.15.167:0/1179388886 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fa67c039060 0x7fa67c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:27.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.144+0000 7fa6937fe640 1 --2- 172.21.15.167:0/1179388886 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fa67c039060 0x7fa67c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa6880098c0 tx=0x7fa688006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:27.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.149+0000 7fa6917fa640 1 -- 172.21.15.167:0/1179388886 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fa68402f300 con 0x7fa694100dd0 2024-06-18T12:58:27.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.327+0000 7fa69a9df640 1 -- 172.21.15.167:0/1179388886 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fa694074610 con 0x7fa67c039060 2024-06-18T12:58:27.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.330+0000 7fa6917fa640 1 -- 172.21.15.167:0/1179388886 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa694074610 con 0x7fa67c039060 2024-06-18T12:58:27.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.333+0000 7fa69a9df640 1 -- 172.21.15.167:0/1179388886 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fa67c039060 msgr2=0x7fa67c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:27.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.333+0000 7fa69a9df640 1 --2- 172.21.15.167:0/1179388886 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fa67c039060 0x7fa67c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa6880098c0 tx=0x7fa688006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:27.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.333+0000 7fa69a9df640 1 -- 172.21.15.167:0/1179388886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694100dd0 msgr2=0x7fa694075660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:27.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.333+0000 7fa69a9df640 1 --2- 172.21.15.167:0/1179388886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694100dd0 0x7fa694075660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa6840057c0 tx=0x7fa684005160 comp rx=0 tx=0).stop 2024-06-18T12:58:27.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.333+0000 7fa69a9df640 1 -- 172.21.15.167:0/1179388886 shutdown_connections 2024-06-18T12:58:27.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.333+0000 7fa69a9df640 1 --2- 172.21.15.167:0/1179388886 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fa67c039060 0x7fa67c03b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:27.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.333+0000 7fa69a9df640 1 --2- 172.21.15.167:0/1179388886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa694100dd0 0x7fa694075660 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:27.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.333+0000 7fa69a9df640 1 -- 172.21.15.167:0/1179388886 >> 172.21.15.167:0/1179388886 conn(0x7fa6940fc5a0 msgr2=0x7fa6940fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:27.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.333+0000 7fa69a9df640 1 -- 172.21.15.167:0/1179388886 shutdown_connections 2024-06-18T12:58:27.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:27.333+0000 7fa69a9df640 1 -- 172.21.15.167:0/1179388886 wait complete. 2024-06-18T12:58:28.178 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi179 2024-06-18T12:58:28.178 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-18T12:58:28.178 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.conf 2024-06-18T12:58:28.198 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-18T12:58:28.198 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-18T12:58:28.256 INFO:tasks.cephadm:Adding host smithi179 to orchestrator... 2024-06-18T12:58:28.257 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch host add smithi179 2024-06-18T12:58:28.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:28 smithi167 ceph-mon[29496]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:58:28.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:28 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:28.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:28 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:58:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:28 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:58:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:28 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:58:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:28 smithi167 ceph-mon[29496]: Updating smithi167:/etc/ceph/ceph.conf 2024-06-18T12:58:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:28 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:28 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:28 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:28.487 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T12:58:29.522 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:29 smithi167 ceph-mon[29496]: Updating smithi167:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T12:58:29.522 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:29 smithi167 ceph-mon[29496]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-06-18T12:58:29.522 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:29 smithi167 ceph-mon[29496]: Updating smithi167:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-18T12:58:29.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.983+0000 7fe356f02640 1 -- 172.21.15.167:0/3073964214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350105d80 msgr2=0x7fe350106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:29.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.983+0000 7fe356f02640 1 --2- 172.21.15.167:0/3073964214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350105d80 0x7fe350106160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe340009880 tx=0x7fe34002a5d0 comp rx=0 tx=0).stop 2024-06-18T12:58:29.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.983+0000 7fe356f02640 1 -- 172.21.15.167:0/3073964214 shutdown_connections 2024-06-18T12:58:29.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.983+0000 7fe356f02640 1 --2- 172.21.15.167:0/3073964214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350105d80 0x7fe350106160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:29.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.983+0000 7fe356f02640 1 -- 172.21.15.167:0/3073964214 >> 172.21.15.167:0/3073964214 conn(0x7fe350075a00 msgr2=0x7fe350075e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:29.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.983+0000 7fe356f02640 1 -- 172.21.15.167:0/3073964214 shutdown_connections 2024-06-18T12:58:29.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.983+0000 7fe356f02640 1 -- 172.21.15.167:0/3073964214 wait complete. 2024-06-18T12:58:29.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.983+0000 7fe356f02640 1 Processor -- start 2024-06-18T12:58:29.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.983+0000 7fe356f02640 1 -- start start 2024-06-18T12:58:29.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.983+0000 7fe356f02640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350105d80 0x7fe350195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:29.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.984+0000 7fe356f02640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe35010a910 con 0x7fe350105d80 2024-06-18T12:58:29.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.984+0000 7fe354c9e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350105d80 0x7fe350195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:29.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.984+0000 7fe354c9e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350105d80 0x7fe350195f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:57702/0 (socket says 172.21.15.167:57702) 2024-06-18T12:58:29.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.984+0000 7fe354c9e640 1 -- 172.21.15.167:0/1003475078 learned_addr learned my addr 172.21.15.167:0/1003475078 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:29.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.984+0000 7fe354c9e640 1 -- 172.21.15.167:0/1003475078 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe350196480 con 0x7fe350105d80 2024-06-18T12:58:29.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.984+0000 7fe354c9e640 1 --2- 172.21.15.167:0/1003475078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350105d80 0x7fe350195f40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe340005ec0 tx=0x7fe3400048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:29.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.984+0000 7fe34dffb640 1 -- 172.21.15.167:0/1003475078 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe34003f070 con 0x7fe350105d80 2024-06-18T12:58:29.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.984+0000 7fe34dffb640 1 -- 172.21.15.167:0/1003475078 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fe340004d10 con 0x7fe350105d80 2024-06-18T12:58:29.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.984+0000 7fe356f02640 1 -- 172.21.15.167:0/1003475078 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe350196710 con 0x7fe350105d80 2024-06-18T12:58:29.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.984+0000 7fe34dffb640 1 -- 172.21.15.167:0/1003475078 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe340035030 con 0x7fe350105d80 2024-06-18T12:58:29.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.984+0000 7fe356f02640 1 -- 172.21.15.167:0/1003475078 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe350196b70 con 0x7fe350105d80 2024-06-18T12:58:29.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.985+0000 7fe34dffb640 1 -- 172.21.15.167:0/1003475078 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fe340042070 con 0x7fe350105d80 2024-06-18T12:58:29.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.985+0000 7fe356f02640 1 -- 172.21.15.167:0/1003475078 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3501024e0 con 0x7fe350105d80 2024-06-18T12:58:29.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.985+0000 7fe34dffb640 1 --2- 172.21.15.167:0/1003475078 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe3380390c0 0x7fe33803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:29.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.986+0000 7fe34dffb640 1 -- 172.21.15.167:0/1003475078 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe34006cd60 con 0x7fe350105d80 2024-06-18T12:58:29.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.986+0000 7fe34ffff640 1 --2- 172.21.15.167:0/1003475078 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe3380390c0 0x7fe33803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:29.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.986+0000 7fe34ffff640 1 --2- 172.21.15.167:0/1003475078 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe3380390c0 0x7fe33803b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe3440098c0 tx=0x7fe344006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:29.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:29.990+0000 7fe34dffb640 1 -- 172.21.15.167:0/1003475078 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fe34003bd50 con 0x7fe350105d80 2024-06-18T12:58:30.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:30.168+0000 7fe356f02640 1 -- 172.21.15.167:0/1003475078 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}) -- 0x7fe3501972a0 con 0x7fe3380390c0 2024-06-18T12:58:30.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:30 smithi167 ceph-mon[29496]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:58:31.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:31 smithi167 ceph-mon[29496]: Deploying cephadm binary to smithi179 2024-06-18T12:58:33.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:33.429+0000 7fe34dffb640 1 -- 172.21.15.167:0/1003475078 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fe3501972a0 con 0x7fe3380390c0 2024-06-18T12:58:33.429 INFO:teuthology.orchestra.run.smithi167.stdout:Added host 'smithi179' with addr '172.21.15.179' 2024-06-18T12:58:33.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:33.431+0000 7fe356f02640 1 -- 172.21.15.167:0/1003475078 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe3380390c0 msgr2=0x7fe33803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:33.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:33.431+0000 7fe356f02640 1 --2- 172.21.15.167:0/1003475078 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe3380390c0 0x7fe33803b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe3440098c0 tx=0x7fe344006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:33.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:33.431+0000 7fe356f02640 1 -- 172.21.15.167:0/1003475078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350105d80 msgr2=0x7fe350195f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:33.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:33.431+0000 7fe356f02640 1 --2- 172.21.15.167:0/1003475078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350105d80 0x7fe350195f40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe340005ec0 tx=0x7fe3400048f0 comp rx=0 tx=0).stop 2024-06-18T12:58:33.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:33.431+0000 7fe356f02640 1 -- 172.21.15.167:0/1003475078 shutdown_connections 2024-06-18T12:58:33.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:33.431+0000 7fe356f02640 1 --2- 172.21.15.167:0/1003475078 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe3380390c0 0x7fe33803b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:33.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:33.431+0000 7fe356f02640 1 --2- 172.21.15.167:0/1003475078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350105d80 0x7fe350195f40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:33.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:33.431+0000 7fe356f02640 1 -- 172.21.15.167:0/1003475078 >> 172.21.15.167:0/1003475078 conn(0x7fe350075a00 msgr2=0x7fe3500ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:33.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:33.432+0000 7fe356f02640 1 -- 172.21.15.167:0/1003475078 shutdown_connections 2024-06-18T12:58:33.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:33.432+0000 7fe356f02640 1 -- 172.21.15.167:0/1003475078 wait complete. 2024-06-18T12:58:34.064 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch host ls --format=json 2024-06-18T12:58:34.299 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T12:58:34.624 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:34.624 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:34 smithi167 ceph-mon[29496]: Added host smithi179 2024-06-18T12:58:34.624 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:58:34.624 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:34 smithi167 ceph-mon[29496]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:34.624 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:34.624 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:35.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.738+0000 7ff34675f640 1 -- 172.21.15.167:0/2797646446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff340101f30 msgr2=0x7ff340102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:35.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.738+0000 7ff34675f640 1 --2- 172.21.15.167:0/2797646446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff340101f30 0x7ff340102310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff330009880 tx=0x7ff33002a670 comp rx=0 tx=0).stop 2024-06-18T12:58:35.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.738+0000 7ff34675f640 1 -- 172.21.15.167:0/2797646446 shutdown_connections 2024-06-18T12:58:35.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.738+0000 7ff34675f640 1 --2- 172.21.15.167:0/2797646446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff340101f30 0x7ff340102310 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:35.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.738+0000 7ff34675f640 1 -- 172.21.15.167:0/2797646446 >> 172.21.15.167:0/2797646446 conn(0x7ff3400fb950 msgr2=0x7ff3400fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:35.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.738+0000 7ff34675f640 1 -- 172.21.15.167:0/2797646446 shutdown_connections 2024-06-18T12:58:35.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.738+0000 7ff34675f640 1 -- 172.21.15.167:0/2797646446 wait complete. 2024-06-18T12:58:35.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.739+0000 7ff34675f640 1 Processor -- start 2024-06-18T12:58:35.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.739+0000 7ff34675f640 1 -- start start 2024-06-18T12:58:35.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.739+0000 7ff34675f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff340101f30 0x7ff340195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:35.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.739+0000 7ff34675f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff34010a900 con 0x7ff340101f30 2024-06-18T12:58:35.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.739+0000 7ff33ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff340101f30 0x7ff340195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:35.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.739+0000 7ff33ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff340101f30 0x7ff340195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51950/0 (socket says 172.21.15.167:51950) 2024-06-18T12:58:35.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.739+0000 7ff33ffff640 1 -- 172.21.15.167:0/4093020273 learned_addr learned my addr 172.21.15.167:0/4093020273 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:35.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.739+0000 7ff33ffff640 1 -- 172.21.15.167:0/4093020273 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3401963f0 con 0x7ff340101f30 2024-06-18T12:58:35.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.740+0000 7ff33ffff640 1 --2- 172.21.15.167:0/4093020273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff340101f30 0x7ff340195eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff330005ec0 tx=0x7ff3300048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:35.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.740+0000 7ff33d7fa640 1 -- 172.21.15.167:0/4093020273 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff33003f070 con 0x7ff340101f30 2024-06-18T12:58:35.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.740+0000 7ff33d7fa640 1 -- 172.21.15.167:0/4093020273 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7ff330004d10 con 0x7ff340101f30 2024-06-18T12:58:35.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.740+0000 7ff34675f640 1 -- 172.21.15.167:0/4093020273 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff340196680 con 0x7ff340101f30 2024-06-18T12:58:35.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.740+0000 7ff33d7fa640 1 -- 172.21.15.167:0/4093020273 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff330035030 con 0x7ff340101f30 2024-06-18T12:58:35.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.740+0000 7ff34675f640 1 -- 172.21.15.167:0/4093020273 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff340196ae0 con 0x7ff340101f30 2024-06-18T12:58:35.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.741+0000 7ff33d7fa640 1 -- 172.21.15.167:0/4093020273 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ff330042070 con 0x7ff340101f30 2024-06-18T12:58:35.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.741+0000 7ff34675f640 1 -- 172.21.15.167:0/4093020273 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff340069f70 con 0x7ff340101f30 2024-06-18T12:58:35.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.741+0000 7ff33d7fa640 1 --2- 172.21.15.167:0/4093020273 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff328039060 0x7ff32803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:35.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.741+0000 7ff33d7fa640 1 -- 172.21.15.167:0/4093020273 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff33006cc20 con 0x7ff340101f30 2024-06-18T12:58:35.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.741+0000 7ff33f7fe640 1 --2- 172.21.15.167:0/4093020273 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff328039060 0x7ff32803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:35.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.741+0000 7ff33f7fe640 1 --2- 172.21.15.167:0/4093020273 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff328039060 0x7ff32803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff3340098c0 tx=0x7ff334006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:35.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.746+0000 7ff33d7fa640 1 -- 172.21.15.167:0/4093020273 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7ff33004c350 con 0x7ff340101f30 2024-06-18T12:58:35.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.925+0000 7ff34675f640 1 -- 172.21.15.167:0/4093020273 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff340197210 con 0x7ff328039060 2024-06-18T12:58:35.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.926+0000 7ff33d7fa640 1 -- 172.21.15.167:0/4093020273 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7ff340197210 con 0x7ff328039060 2024-06-18T12:58:35.925 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T12:58:35.926 INFO:teuthology.orchestra.run.smithi167.stdout:[{"addr": "172.21.15.167", "hostname": "smithi167", "labels": [], "status": ""}, {"addr": "172.21.15.179", "hostname": "smithi179", "labels": [], "status": ""}] 2024-06-18T12:58:35.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.928+0000 7ff34675f640 1 -- 172.21.15.167:0/4093020273 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff328039060 msgr2=0x7ff32803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:35.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.928+0000 7ff34675f640 1 --2- 172.21.15.167:0/4093020273 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff328039060 0x7ff32803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff3340098c0 tx=0x7ff334006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:35.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.928+0000 7ff34675f640 1 -- 172.21.15.167:0/4093020273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff340101f30 msgr2=0x7ff340195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.928+0000 7ff34675f640 1 --2- 172.21.15.167:0/4093020273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff340101f30 0x7ff340195eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff330005ec0 tx=0x7ff3300048f0 comp rx=0 tx=0).stop 2024-06-18T12:58:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.928+0000 7ff34675f640 1 -- 172.21.15.167:0/4093020273 shutdown_connections 2024-06-18T12:58:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.928+0000 7ff34675f640 1 --2- 172.21.15.167:0/4093020273 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff328039060 0x7ff32803b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.928+0000 7ff34675f640 1 --2- 172.21.15.167:0/4093020273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff340101f30 0x7ff340195eb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.928+0000 7ff34675f640 1 -- 172.21.15.167:0/4093020273 >> 172.21.15.167:0/4093020273 conn(0x7ff3400fb950 msgr2=0x7ff340108300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.928+0000 7ff34675f640 1 -- 172.21.15.167:0/4093020273 shutdown_connections 2024-06-18T12:58:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:35.928+0000 7ff34675f640 1 -- 172.21.15.167:0/4093020273 wait complete. 2024-06-18T12:58:36.216 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:35 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:36.217 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:35 smithi167 ceph-mon[29496]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:36.547 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-18T12:58:36.547 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd crush tunables default 2024-06-18T12:58:36.780 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T12:58:37.062 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:36 smithi167 ceph-mon[29496]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-18T12:58:37.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:37 smithi167 ceph-mon[29496]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:37.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:37 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:37.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:37 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:37.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:37 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:37.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:37 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:37.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:37 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-06-18T12:58:37.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:37 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:58:37.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:37 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:58:38.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.188+0000 7fc3f63a3640 1 -- 172.21.15.167:0/36375294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f0101f30 msgr2=0x7fc3f0102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:38.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.188+0000 7fc3f63a3640 1 --2- 172.21.15.167:0/36375294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f0101f30 0x7fc3f0102310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009880 tx=0x7fc3e002a670 comp rx=0 tx=0).stop 2024-06-18T12:58:38.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.188+0000 7fc3f63a3640 1 -- 172.21.15.167:0/36375294 shutdown_connections 2024-06-18T12:58:38.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.188+0000 7fc3f63a3640 1 --2- 172.21.15.167:0/36375294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f0101f30 0x7fc3f0102310 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:38.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.188+0000 7fc3f63a3640 1 -- 172.21.15.167:0/36375294 >> 172.21.15.167:0/36375294 conn(0x7fc3f00fb950 msgr2=0x7fc3f00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:38.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.188+0000 7fc3f63a3640 1 -- 172.21.15.167:0/36375294 shutdown_connections 2024-06-18T12:58:38.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.188+0000 7fc3f63a3640 1 -- 172.21.15.167:0/36375294 wait complete. 2024-06-18T12:58:38.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.188+0000 7fc3f63a3640 1 Processor -- start 2024-06-18T12:58:38.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.188+0000 7fc3f63a3640 1 -- start start 2024-06-18T12:58:38.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.188+0000 7fc3f63a3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f0101f30 0x7fc3f0195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:38.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.188+0000 7fc3f63a3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f010a900 con 0x7fc3f0101f30 2024-06-18T12:58:38.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.189+0000 7fc3effff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f0101f30 0x7fc3f0195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:38.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.189+0000 7fc3effff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f0101f30 0x7fc3f0195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51974/0 (socket says 172.21.15.167:51974) 2024-06-18T12:58:38.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.189+0000 7fc3effff640 1 -- 172.21.15.167:0/2023705182 learned_addr learned my addr 172.21.15.167:0/2023705182 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:58:38.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.189+0000 7fc3effff640 1 -- 172.21.15.167:0/2023705182 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f01963e0 con 0x7fc3f0101f30 2024-06-18T12:58:38.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.189+0000 7fc3effff640 1 --2- 172.21.15.167:0/2023705182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f0101f30 0x7fc3f0195ea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0005ab0 tx=0x7fc3e00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:38.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.189+0000 7fc3edffb640 1 -- 172.21.15.167:0/2023705182 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3e003f070 con 0x7fc3f0101f30 2024-06-18T12:58:38.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.189+0000 7fc3f63a3640 1 -- 172.21.15.167:0/2023705182 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f0196670 con 0x7fc3f0101f30 2024-06-18T12:58:38.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.189+0000 7fc3edffb640 1 -- 172.21.15.167:0/2023705182 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fc3e0004cc0 con 0x7fc3f0101f30 2024-06-18T12:58:38.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.189+0000 7fc3edffb640 1 -- 172.21.15.167:0/2023705182 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3e0035030 con 0x7fc3f0101f30 2024-06-18T12:58:38.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.189+0000 7fc3f63a3640 1 -- 172.21.15.167:0/2023705182 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f0196ad0 con 0x7fc3f0101f30 2024-06-18T12:58:38.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.190+0000 7fc3edffb640 1 -- 172.21.15.167:0/2023705182 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc3e0042070 con 0x7fc3f0101f30 2024-06-18T12:58:38.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.190+0000 7fc3f63a3640 1 -- 172.21.15.167:0/2023705182 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f0069f70 con 0x7fc3f0101f30 2024-06-18T12:58:38.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.190+0000 7fc3edffb640 1 --2- 172.21.15.167:0/2023705182 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc3dc039060 0x7fc3dc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:38.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.190+0000 7fc3edffb640 1 -- 172.21.15.167:0/2023705182 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc3e006cd60 con 0x7fc3f0101f30 2024-06-18T12:58:38.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.191+0000 7fc3e7fff640 1 --2- 172.21.15.167:0/2023705182 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc3dc039060 0x7fc3dc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:38.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.191+0000 7fc3e7fff640 1 --2- 172.21.15.167:0/2023705182 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc3dc039060 0x7fc3dc03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80098c0 tx=0x7fc3d8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:38.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.195+0000 7fc3edffb640 1 -- 172.21.15.167:0/2023705182 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fc3e003bd30 con 0x7fc3f0101f30 2024-06-18T12:58:38.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.376+0000 7fc3f63a3640 1 -- 172.21.15.167:0/2023705182 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc3f01023c0 con 0x7fc3f0101f30 2024-06-18T12:58:38.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.792+0000 7fc3edffb640 1 -- 172.21.15.167:0/2023705182 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fc3f01023c0 con 0x7fc3f0101f30 2024-06-18T12:58:38.856 INFO:teuthology.orchestra.run.smithi167.stderr:adjusted tunables profile to default 2024-06-18T12:58:38.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.794+0000 7fc3f63a3640 1 -- 172.21.15.167:0/2023705182 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc3dc039060 msgr2=0x7fc3dc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:38.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.794+0000 7fc3f63a3640 1 --2- 172.21.15.167:0/2023705182 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc3dc039060 0x7fc3dc03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80098c0 tx=0x7fc3d8006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:38.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.794+0000 7fc3f63a3640 1 -- 172.21.15.167:0/2023705182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f0101f30 msgr2=0x7fc3f0195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:38.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.794+0000 7fc3f63a3640 1 --2- 172.21.15.167:0/2023705182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f0101f30 0x7fc3f0195ea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0005ab0 tx=0x7fc3e00048f0 comp rx=0 tx=0).stop 2024-06-18T12:58:38.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.794+0000 7fc3f63a3640 1 -- 172.21.15.167:0/2023705182 shutdown_connections 2024-06-18T12:58:38.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.794+0000 7fc3f63a3640 1 --2- 172.21.15.167:0/2023705182 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc3dc039060 0x7fc3dc03b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:38.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.794+0000 7fc3f63a3640 1 --2- 172.21.15.167:0/2023705182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f0101f30 0x7fc3f0195ea0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:38.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.794+0000 7fc3f63a3640 1 -- 172.21.15.167:0/2023705182 >> 172.21.15.167:0/2023705182 conn(0x7fc3f00fb950 msgr2=0x7fc3f0105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:38.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.794+0000 7fc3f63a3640 1 -- 172.21.15.167:0/2023705182 shutdown_connections 2024-06-18T12:58:38.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:58:38.794+0000 7fc3f63a3640 1 -- 172.21.15.167:0/2023705182 wait complete. 2024-06-18T12:58:39.090 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:38 smithi167 ceph-mon[29496]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-18T12:58:39.090 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:38 smithi167 ceph-mon[29496]: Updating smithi179:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T12:58:39.090 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:38 smithi167 ceph-mon[29496]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-06-18T12:58:39.090 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:38 smithi167 ceph-mon[29496]: Updating smithi179:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-18T12:58:39.090 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:38 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2023705182' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-18T12:58:39.090 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:38 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:39.090 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:38 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:39.090 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:38 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:39.385 INFO:tasks.cephadm:Adding mon.a on smithi167 2024-06-18T12:58:39.385 INFO:tasks.cephadm:Adding mon.c on smithi167 2024-06-18T12:58:39.385 INFO:tasks.cephadm:Adding mon.b on smithi179 2024-06-18T12:58:39.385 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch apply mon '3;smithi167:172.21.15.167=a;smithi167:[v2:172.21.15.167:3301,v1:172.21.15.167:6790]=c;smithi179:172.21.15.179=b' 2024-06-18T12:58:39.617 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T12:58:40.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2023705182' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-18T12:58:40.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:39 smithi167 ceph-mon[29496]: osdmap e4: 0 total, 0 up, 0 in 2024-06-18T12:58:40.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:39 smithi167 ceph-mon[29496]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.017+0000 7f1e5c185640 1 -- 172.21.15.179:0/1653370003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e54100140 msgr2=0x7f1e54100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.017+0000 7f1e5c185640 1 --2- 172.21.15.179:0/1653370003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e54100140 0x7f1e54100520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009880 tx=0x7f1e5002a5d0 comp rx=0 tx=0).stop 2024-06-18T12:58:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.017+0000 7f1e5c185640 1 -- 172.21.15.179:0/1653370003 shutdown_connections 2024-06-18T12:58:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.017+0000 7f1e5c185640 1 --2- 172.21.15.179:0/1653370003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e54100140 0x7f1e54100520 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.017+0000 7f1e5c185640 1 -- 172.21.15.179:0/1653370003 >> 172.21.15.179:0/1653370003 conn(0x7f1e54075a00 msgr2=0x7f1e54075e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.017+0000 7f1e5c185640 1 -- 172.21.15.179:0/1653370003 shutdown_connections 2024-06-18T12:58:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.017+0000 7f1e5c185640 1 -- 172.21.15.179:0/1653370003 wait complete. 2024-06-18T12:58:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.018+0000 7f1e5c185640 1 Processor -- start 2024-06-18T12:58:41.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.018+0000 7f1e5c185640 1 -- start start 2024-06-18T12:58:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.018+0000 7f1e5c185640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e54100140 0x7f1e54194b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.018+0000 7f1e5c185640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1e541097c0 con 0x7f1e54100140 2024-06-18T12:58:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.018+0000 7f1e59f21640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e54100140 0x7f1e54194b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.018+0000 7f1e59f21640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e54100140 0x7f1e54194b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.179:50814/0 (socket says 172.21.15.179:50814) 2024-06-18T12:58:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.018+0000 7f1e59f21640 1 -- 172.21.15.179:0/2675643394 learned_addr learned my addr 172.21.15.179:0/2675643394 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T12:58:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.019+0000 7f1e59f21640 1 -- 172.21.15.179:0/2675643394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e54069c70 con 0x7f1e54100140 2024-06-18T12:58:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.019+0000 7f1e59f21640 1 --2- 172.21.15.179:0/2675643394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e54100140 0x7f1e54194b20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1e50005ab0 tx=0x7f1e500048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.019+0000 7f1e4affd640 1 -- 172.21.15.179:0/2675643394 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e5003f070 con 0x7f1e54100140 2024-06-18T12:58:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.019+0000 7f1e5c185640 1 -- 172.21.15.179:0/2675643394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e54195060 con 0x7f1e54100140 2024-06-18T12:58:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.019+0000 7f1e4affd640 1 -- 172.21.15.179:0/2675643394 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f1e50004cc0 con 0x7f1e54100140 2024-06-18T12:58:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.019+0000 7f1e4affd640 1 -- 172.21.15.179:0/2675643394 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e50035030 con 0x7f1e54100140 2024-06-18T12:58:41.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.019+0000 7f1e5c185640 1 -- 172.21.15.179:0/2675643394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e54195520 con 0x7f1e54100140 2024-06-18T12:58:41.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.020+0000 7f1e4affd640 1 -- 172.21.15.179:0/2675643394 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1e50042070 con 0x7f1e54100140 2024-06-18T12:58:41.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.020+0000 7f1e5c185640 1 -- 172.21.15.179:0/2675643394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e541013f0 con 0x7f1e54100140 2024-06-18T12:58:41.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.020+0000 7f1e4affd640 1 --2- 172.21.15.179:0/2675643394 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f1e40039060 0x7f1e4003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:41.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.020+0000 7f1e4affd640 1 -- 172.21.15.179:0/2675643394 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1e5006cc10 con 0x7f1e54100140 2024-06-18T12:58:41.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.021+0000 7f1e59720640 1 --2- 172.21.15.179:0/2675643394 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f1e40039060 0x7f1e4003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:41.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.021+0000 7f1e59720640 1 --2- 172.21.15.179:0/2675643394 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f1e40039060 0x7f1e4003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1e440098c0 tx=0x7f1e44006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:41.025 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.026+0000 7f1e4affd640 1 -- 172.21.15.179:0/2675643394 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f1e5004c350 con 0x7f1e54100140 2024-06-18T12:58:41.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.214+0000 7f1e5c185640 1 -- 172.21.15.179:0/2675643394 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi167:172.21.15.167=a;smithi167:[v2:172.21.15.167:3301,v1:172.21.15.167:6790]=c;smithi179:172.21.15.179=b", "target": ["mon-mgr", ""]}) -- 0x7f1e54069460 con 0x7f1e40039060 2024-06-18T12:58:41.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.220+0000 7f1e4affd640 1 -- 172.21.15.179:0/2675643394 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1e54069460 con 0x7f1e40039060 2024-06-18T12:58:41.220 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mon update... 2024-06-18T12:58:41.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.224+0000 7f1e5c185640 1 -- 172.21.15.179:0/2675643394 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f1e40039060 msgr2=0x7f1e4003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:41.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.224+0000 7f1e5c185640 1 --2- 172.21.15.179:0/2675643394 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f1e40039060 0x7f1e4003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1e440098c0 tx=0x7f1e44006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:41.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.224+0000 7f1e5c185640 1 -- 172.21.15.179:0/2675643394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e54100140 msgr2=0x7f1e54194b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:41.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.224+0000 7f1e5c185640 1 --2- 172.21.15.179:0/2675643394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e54100140 0x7f1e54194b20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1e50005ab0 tx=0x7f1e500048f0 comp rx=0 tx=0).stop 2024-06-18T12:58:41.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.224+0000 7f1e5c185640 1 -- 172.21.15.179:0/2675643394 shutdown_connections 2024-06-18T12:58:41.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.224+0000 7f1e5c185640 1 --2- 172.21.15.179:0/2675643394 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f1e40039060 0x7f1e4003b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:41.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.224+0000 7f1e5c185640 1 --2- 172.21.15.179:0/2675643394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e54100140 0x7f1e54194b20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:41.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.224+0000 7f1e5c185640 1 -- 172.21.15.179:0/2675643394 >> 172.21.15.179:0/2675643394 conn(0x7f1e54075a00 msgr2=0x7f1e540ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:41.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.224+0000 7f1e5c185640 1 -- 172.21.15.179:0/2675643394 shutdown_connections 2024-06-18T12:58:41.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:41.224+0000 7f1e5c185640 1 -- 172.21.15.179:0/2675643394 wait complete. 2024-06-18T12:58:41.859 DEBUG:teuthology.orchestra.run.smithi167:mon.c> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.c.service 2024-06-18T12:58:41.862 DEBUG:teuthology.orchestra.run.smithi179:mon.b> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.b.service 2024-06-18T12:58:41.864 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-18T12:58:41.864 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-18T12:58:42.412 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T12:58:42.474 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:42 smithi167 ceph-mon[29496]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi167:172.21.15.167=a;smithi167:[v2:172.21.15.167:3301,v1:172.21.15.167:6790]=c;smithi179:172.21.15.179=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:58:42.474 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:42 smithi167 ceph-mon[29496]: Saving service mon spec with placement smithi167:172.21.15.167=a;smithi167:[v2:172.21.15.167:3301,v1:172.21.15.167:6790]=c;smithi179:172.21.15.179=b;count:3 2024-06-18T12:58:42.474 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:42 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:42.474 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:42 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:58:42.474 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:42 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:58:42.474 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:42 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:58:42.474 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:42 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:42.474 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:42 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-18T12:58:42.474 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:42 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:58:42.474 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:42 smithi167 ceph-mon[29496]: Deploying daemon mon.b on smithi179 2024-06-18T12:58:42.474 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:42 smithi167 ceph-mon[29496]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:43.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:43 smithi167 ceph-mon[29496]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:45.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.427+0000 7fcd2a964640 1 -- 172.21.15.179:0/1543053718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd24101f30 msgr2=0x7fcd24102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:45.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.427+0000 7fcd2a964640 1 --2- 172.21.15.179:0/1543053718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd24101f30 0x7fcd24102310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009880 tx=0x7fcd1402a670 comp rx=0 tx=0).stop 2024-06-18T12:58:45.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.427+0000 7fcd2a964640 1 -- 172.21.15.179:0/1543053718 shutdown_connections 2024-06-18T12:58:45.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.427+0000 7fcd2a964640 1 --2- 172.21.15.179:0/1543053718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd24101f30 0x7fcd24102310 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:45.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.427+0000 7fcd2a964640 1 -- 172.21.15.179:0/1543053718 >> 172.21.15.179:0/1543053718 conn(0x7fcd240fb950 msgr2=0x7fcd240fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:45.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.427+0000 7fcd2a964640 1 -- 172.21.15.179:0/1543053718 shutdown_connections 2024-06-18T12:58:45.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.427+0000 7fcd2a964640 1 -- 172.21.15.179:0/1543053718 wait complete. 2024-06-18T12:58:45.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.427+0000 7fcd2a964640 1 Processor -- start 2024-06-18T12:58:45.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.428+0000 7fcd2a964640 1 -- start start 2024-06-18T12:58:45.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.428+0000 7fcd2a964640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd24101f30 0x7fcd24195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:45.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.428+0000 7fcd2a964640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2410a900 con 0x7fcd24101f30 2024-06-18T12:58:45.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.428+0000 7fcd23fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd24101f30 0x7fcd24195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:45.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.428+0000 7fcd23fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd24101f30 0x7fcd24195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.179:50830/0 (socket says 172.21.15.179:50830) 2024-06-18T12:58:45.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.428+0000 7fcd23fff640 1 -- 172.21.15.179:0/376152029 learned_addr learned my addr 172.21.15.179:0/376152029 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T12:58:45.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.429+0000 7fcd23fff640 1 -- 172.21.15.179:0/376152029 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd241963e0 con 0x7fcd24101f30 2024-06-18T12:58:45.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.429+0000 7fcd23fff640 1 --2- 172.21.15.179:0/376152029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd24101f30 0x7fcd24195ea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcd14005ab0 tx=0x7fcd140048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:45.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.429+0000 7fcd217fa640 1 -- 172.21.15.179:0/376152029 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd1403f070 con 0x7fcd24101f30 2024-06-18T12:58:45.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.429+0000 7fcd2a964640 1 -- 172.21.15.179:0/376152029 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd24196670 con 0x7fcd24101f30 2024-06-18T12:58:45.432 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.429+0000 7fcd217fa640 1 -- 172.21.15.179:0/376152029 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fcd14004cc0 con 0x7fcd24101f30 2024-06-18T12:58:45.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.429+0000 7fcd2a964640 1 -- 172.21.15.179:0/376152029 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd24196ad0 con 0x7fcd24101f30 2024-06-18T12:58:45.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.429+0000 7fcd217fa640 1 -- 172.21.15.179:0/376152029 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd14035030 con 0x7fcd24101f30 2024-06-18T12:58:45.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.430+0000 7fcd217fa640 1 -- 172.21.15.179:0/376152029 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fcd14042070 con 0x7fcd24101f30 2024-06-18T12:58:45.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.430+0000 7fcd2a964640 1 -- 172.21.15.179:0/376152029 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd24069f70 con 0x7fcd24101f30 2024-06-18T12:58:45.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.430+0000 7fcd217fa640 1 --2- 172.21.15.179:0/376152029 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcd0c0390c0 0x7fcd0c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:45.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.430+0000 7fcd217fa640 1 -- 172.21.15.179:0/376152029 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcd1406cda0 con 0x7fcd24101f30 2024-06-18T12:58:45.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.430+0000 7fcd237fe640 1 --2- 172.21.15.179:0/376152029 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcd0c0390c0 0x7fcd0c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:45.433 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.431+0000 7fcd237fe640 1 --2- 172.21.15.179:0/376152029 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcd0c0390c0 0x7fcd0c03b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcd180098c0 tx=0x7fcd18006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:45.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.435+0000 7fcd217fa640 1 -- 172.21.15.179:0/376152029 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fcd1403bdb0 con 0x7fcd24101f30 2024-06-18T12:58:45.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.689+0000 7fcd2a964640 1 -- 172.21.15.179:0/376152029 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcd241023c0 con 0x7fcd24101f30 2024-06-18T12:58:45.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.689+0000 7fcd217fa640 1 -- 172.21.15.179:0/376152029 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fcd241023c0 con 0x7fcd24101f30 2024-06-18T12:58:45.690 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-06-18T12:58:45.691 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:58:45.691 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"3434f1e2-2d72-11ef-bc9d-c7b262605968","modified":"2024-06-18T12:57:16.004836Z","created":"2024-06-18T12:57:16.004836Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-18T12:58:45.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.692+0000 7fcd2a964640 1 -- 172.21.15.179:0/376152029 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcd0c0390c0 msgr2=0x7fcd0c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:45.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.692+0000 7fcd2a964640 1 --2- 172.21.15.179:0/376152029 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcd0c0390c0 0x7fcd0c03b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcd180098c0 tx=0x7fcd18006db0 comp rx=0 tx=0).stop 2024-06-18T12:58:45.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.692+0000 7fcd2a964640 1 -- 172.21.15.179:0/376152029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd24101f30 msgr2=0x7fcd24195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:45.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.692+0000 7fcd2a964640 1 --2- 172.21.15.179:0/376152029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd24101f30 0x7fcd24195ea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcd14005ab0 tx=0x7fcd140048f0 comp rx=0 tx=0).stop 2024-06-18T12:58:45.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.692+0000 7fcd2a964640 1 -- 172.21.15.179:0/376152029 shutdown_connections 2024-06-18T12:58:45.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.692+0000 7fcd2a964640 1 --2- 172.21.15.179:0/376152029 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcd0c0390c0 0x7fcd0c03b580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:45.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.692+0000 7fcd2a964640 1 --2- 172.21.15.179:0/376152029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd24101f30 0x7fcd24195ea0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:45.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.692+0000 7fcd2a964640 1 -- 172.21.15.179:0/376152029 >> 172.21.15.179:0/376152029 conn(0x7fcd240fb950 msgr2=0x7fcd24105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:45.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.692+0000 7fcd2a964640 1 -- 172.21.15.179:0/376152029 shutdown_connections 2024-06-18T12:58:45.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:45.692+0000 7fcd2a964640 1 -- 172.21.15.179:0/376152029 wait complete. 2024-06-18T12:58:45.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:45 smithi167 ceph-mon[29496]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:47.627 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-18T12:58:47.627 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-18T12:58:47.868 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T12:58:51.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:51 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:58:51.437 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T12:58:51.437 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: mon.a calling monitor election 2024-06-18T12:58:51.437 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:51.437 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:51.437 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:51.437 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:51.437 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: mon.b calling monitor election 2024-06-18T12:58:51.438 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:51.438 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:51.438 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:51.438 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:51.438 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-06-18T12:58:51.438 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: monmap epoch 2 2024-06-18T12:58:51.438 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:58:51.438 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: last_changed 2024-06-18T12:58:45.999439+0000 2024-06-18T12:58:51.438 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: created 2024-06-18T12:57:16.004836+0000 2024-06-18T12:58:51.438 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: min_mon_release 19 (squid) 2024-06-18T12:58:51.438 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: election_strategy: 1 2024-06-18T12:58:51.438 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: 0: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.a 2024-06-18T12:58:51.438 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.b 2024-06-18T12:58:51.439 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: fsmap 2024-06-18T12:58:51.439 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: osdmap e4: 0 total, 0 up, 0 in 2024-06-18T12:58:51.439 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: mgrmap e13: y(active, since 37s) 2024-06-18T12:58:51.439 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: overall HEALTH_OK 2024-06-18T12:58:51.439 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:51.439 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:51.439 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:51.439 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-18T12:58:51.439 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:58:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:52 smithi179 ceph-mon[33312]: Deploying daemon mon.c on smithi167 2024-06-18T12:58:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:52 smithi179 ceph-mon[33312]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:52 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:52.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:52 smithi167 ceph-mon[29496]: Deploying daemon mon.c on smithi167 2024-06-18T12:58:52.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:52 smithi167 ceph-mon[29496]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:52.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:52 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:52.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.386+0000 7fed68a5e640 1 -- 172.21.15.179:0/2855972078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640fff70 msgr2=0x7fed50005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:52.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.386+0000 7fed68a5e640 1 --2- 172.21.15.179:0/2855972078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640fff70 0x7fed50005580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fed4c011c50 tx=0x7fed4c02be50 comp rx=0 tx=0).stop 2024-06-18T12:58:52.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.386+0000 7fed68a5e640 1 -- 172.21.15.179:0/2855972078 shutdown_connections 2024-06-18T12:58:52.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.386+0000 7fed68a5e640 1 --2- 172.21.15.179:0/2855972078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640fff70 0x7fed50005580 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:52.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.386+0000 7fed68a5e640 1 -- 172.21.15.179:0/2855972078 >> 172.21.15.179:0/2855972078 conn(0x7fed640fb9d0 msgr2=0x7fed640fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:52.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.386+0000 7fed68a5e640 1 -- 172.21.15.179:0/2855972078 shutdown_connections 2024-06-18T12:58:52.387 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.386+0000 7fed68a5e640 1 -- 172.21.15.179:0/2855972078 wait complete. 2024-06-18T12:58:52.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.388+0000 7fed68a5e640 1 Processor -- start 2024-06-18T12:58:52.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.388+0000 7fed68a5e640 1 -- start start 2024-06-18T12:58:52.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.388+0000 7fed68a5e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640fff70 0x7fed64193d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:52.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.388+0000 7fed68a5e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed64194280 0x7fed64198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:52.388 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.388+0000 7fed68a5e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed6406a2e0 con 0x7fed640fff70 2024-06-18T12:58:52.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.388+0000 7fed68a5e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fed6406a160 con 0x7fed64194280 2024-06-18T12:58:52.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.388+0000 7fed61d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed64194280 0x7fed64198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:52.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.388+0000 7fed61d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed64194280 0x7fed64198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:45752/0 (socket says 172.21.15.179:45752) 2024-06-18T12:58:52.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.388+0000 7fed61d9b640 1 -- 172.21.15.179:0/3577703307 learned_addr learned my addr 172.21.15.179:0/3577703307 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T12:58:52.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.388+0000 7fed6259c640 1 --2- 172.21.15.179:0/3577703307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640fff70 0x7fed64193d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:52.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.388+0000 7fed61d9b640 1 -- 172.21.15.179:0/3577703307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed64194280 msgr2=0x7fed64198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-18T12:58:52.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.388+0000 7fed61d9b640 1 -- 172.21.15.179:0/3577703307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed64194280 msgr2=0x7fed64198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-18T12:58:52.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.388+0000 7fed61d9b640 1 --2- 172.21.15.179:0/3577703307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed64194280 0x7fed64198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-18T12:58:52.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.388+0000 7fed61d9b640 1 --2- 172.21.15.179:0/3577703307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed64194280 0x7fed64198610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-18T12:58:52.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.389+0000 7fed6259c640 1 -- 172.21.15.179:0/3577703307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed64194280 msgr2=0x7fed64198610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-18T12:58:52.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.389+0000 7fed6259c640 1 --2- 172.21.15.179:0/3577703307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed64194280 0x7fed64198610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:52.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.389+0000 7fed6259c640 1 -- 172.21.15.179:0/3577703307 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed641948a0 con 0x7fed640fff70 2024-06-18T12:58:52.389 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.389+0000 7fed6259c640 1 --2- 172.21.15.179:0/3577703307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640fff70 0x7fed64193d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed4c005f50 tx=0x7fed4c010600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:52.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.389+0000 7fed5b7fe640 1 -- 172.21.15.179:0/3577703307 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fed4c005040 con 0x7fed640fff70 2024-06-18T12:58:52.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.389+0000 7fed5b7fe640 1 -- 172.21.15.179:0/3577703307 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fed4c043070 con 0x7fed640fff70 2024-06-18T12:58:52.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.389+0000 7fed5b7fe640 1 -- 172.21.15.179:0/3577703307 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fed4c004d00 con 0x7fed640fff70 2024-06-18T12:58:52.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.389+0000 7fed68a5e640 1 -- 172.21.15.179:0/3577703307 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed64198b50 con 0x7fed640fff70 2024-06-18T12:58:52.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.389+0000 7fed68a5e640 1 -- 172.21.15.179:0/3577703307 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed64199090 con 0x7fed640fff70 2024-06-18T12:58:52.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.390+0000 7fed5b7fe640 1 -- 172.21.15.179:0/3577703307 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fed4c048460 con 0x7fed640fff70 2024-06-18T12:58:52.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.390+0000 7fed5b7fe640 1 --2- 172.21.15.179:0/3577703307 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fed44038e00 0x7fed4403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:52.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.390+0000 7fed5b7fe640 1 -- 172.21.15.179:0/3577703307 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fed4c03b040 con 0x7fed640fff70 2024-06-18T12:58:52.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.391+0000 7fed61d9b640 1 --2- 172.21.15.179:0/3577703307 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fed44038e00 0x7fed4403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:52.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.391+0000 7fed68a5e640 1 -- 172.21.15.179:0/3577703307 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed64101450 con 0x7fed640fff70 2024-06-18T12:58:52.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.391+0000 7fed61d9b640 1 --2- 172.21.15.179:0/3577703307 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fed44038e00 0x7fed4403b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fed54009100 tx=0x7fed54008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:52.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.396+0000 7fed5b7fe640 1 -- 172.21.15.179:0/3577703307 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fed4c0465c0 con 0x7fed640fff70 2024-06-18T12:58:52.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.645+0000 7fed68a5e640 1 -- 172.21.15.179:0/3577703307 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fed6418d770 con 0x7fed640fff70 2024-06-18T12:58:52.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.646+0000 7fed5b7fe640 1 -- 172.21.15.179:0/3577703307 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fed4c046360 con 0x7fed640fff70 2024-06-18T12:58:52.646 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:58:52.646 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":2,"fsid":"3434f1e2-2d72-11ef-bc9d-c7b262605968","modified":"2024-06-18T12:58:45.999439Z","created":"2024-06-18T12:57:16.004836Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-18T12:58:52.646 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 2 2024-06-18T12:58:52.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.649+0000 7fed68a5e640 1 -- 172.21.15.179:0/3577703307 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fed44038e00 msgr2=0x7fed4403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:52.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.649+0000 7fed68a5e640 1 --2- 172.21.15.179:0/3577703307 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fed44038e00 0x7fed4403b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fed54009100 tx=0x7fed54008040 comp rx=0 tx=0).stop 2024-06-18T12:58:52.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.649+0000 7fed68a5e640 1 -- 172.21.15.179:0/3577703307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640fff70 msgr2=0x7fed64193d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:52.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.649+0000 7fed68a5e640 1 --2- 172.21.15.179:0/3577703307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640fff70 0x7fed64193d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed4c005f50 tx=0x7fed4c010600 comp rx=0 tx=0).stop 2024-06-18T12:58:52.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.649+0000 7fed68a5e640 1 -- 172.21.15.179:0/3577703307 shutdown_connections 2024-06-18T12:58:52.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.649+0000 7fed68a5e640 1 --2- 172.21.15.179:0/3577703307 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fed44038e00 0x7fed4403b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:52.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.649+0000 7fed68a5e640 1 --2- 172.21.15.179:0/3577703307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed64194280 0x7fed64198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:52.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.649+0000 7fed68a5e640 1 --2- 172.21.15.179:0/3577703307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed640fff70 0x7fed64193d40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:52.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.649+0000 7fed68a5e640 1 -- 172.21.15.179:0/3577703307 >> 172.21.15.179:0/3577703307 conn(0x7fed640fb9d0 msgr2=0x7fed640fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:52.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.649+0000 7fed68a5e640 1 -- 172.21.15.179:0/3577703307 shutdown_connections 2024-06-18T12:58:52.649 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:52.649+0000 7fed68a5e640 1 -- 172.21.15.179:0/3577703307 wait complete. 2024-06-18T12:58:53.207 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.179:0/3577703307' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-18T12:58:53.463 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.179:0/3577703307' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-18T12:58:53.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:53 smithi167 systemd[1]: Starting Ceph mon.c for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T12:58:53.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:53 smithi167 podman[37680]: 2024-06-18 12:58:53.899495178 +0000 UTC m=+0.110261632 container create de3cc70740a845bc58ce314fbe054ccc5071896eddc5c83a101ebf1acfb521e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240612, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=squid-3781e3a, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0) 2024-06-18T12:58:53.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:53 smithi167 podman[37680]: 2024-06-18 12:58:53.806474849 +0000 UTC m=+0.017241321 image pull af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314 2024-06-18T12:58:54.236 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-18T12:58:54.236 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-18T12:58:54.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:54 smithi179 ceph-mon[33312]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:54.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[29496]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:54.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 podman[37680]: 2024-06-18 12:58:54.118582328 +0000 UTC m=+0.329348783 container init de3cc70740a845bc58ce314fbe054ccc5071896eddc5c83a101ebf1acfb521e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-c, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, RELEASE=squid-3781e3a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-06-18T12:58:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 podman[37680]: 2024-06-18 12:58:54.121790628 +0000 UTC m=+0.332557085 container start de3cc70740a845bc58ce314fbe054ccc5071896eddc5c83a101ebf1acfb521e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3781e3a, org.label-schema.build-date=20240612, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0) 2024-06-18T12:58:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: set uid:gid to 167:167 (ceph:ceph) 2024-06-18T12:58:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: ceph version 19.0.0-3124-g3781e3a5 (3781e3a592387d16e8020f5db99cdee4198f6d00) squid (rc), process ceph-mon, pid 2 2024-06-18T12:58:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: pidfile_write: ignore empty --pid-file 2024-06-18T12:58:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: load: jerasure load: lrc 2024-06-18T12:58:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: RocksDB version: 7.9.2 2024-06-18T12:58:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Git sha 0 2024-06-18T12:58:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Compile date 2024-06-17 15:21:02 2024-06-18T12:58:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: DB SUMMARY 2024-06-18T12:58:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: DB Session ID: TSIWY9H61FIJZG3IQDST 2024-06-18T12:58:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: CURRENT file: CURRENT 2024-06-18T12:58:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: IDENTITY file: IDENTITY 2024-06-18T12:58:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-06-18T12:58:54.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-06-18T12:58:54.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-06-18T12:58:54.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.error_if_exists: 0 2024-06-18T12:58:54.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.create_if_missing: 0 2024-06-18T12:58:54.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.paranoid_checks: 1 2024-06-18T12:58:54.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-18T12:58:54.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-18T12:58:54.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-18T12:58:54.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.env: 0x560a5da94740 2024-06-18T12:58:54.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.fs: PosixFileSystem 2024-06-18T12:58:54.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.info_log: 0x560a5fe89000 2024-06-18T12:58:54.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_file_opening_threads: 16 2024-06-18T12:58:54.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.statistics: (nil) 2024-06-18T12:58:54.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.use_fsync: 0 2024-06-18T12:58:54.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_log_file_size: 0 2024-06-18T12:58:54.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-18T12:58:54.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-18T12:58:54.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.keep_log_file_num: 1000 2024-06-18T12:58:54.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.recycle_log_file_num: 0 2024-06-18T12:58:54.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.allow_fallocate: 1 2024-06-18T12:58:54.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.allow_mmap_reads: 0 2024-06-18T12:58:54.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.allow_mmap_writes: 0 2024-06-18T12:58:54.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.use_direct_reads: 0 2024-06-18T12:58:54.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-18T12:58:54.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.create_missing_column_families: 0 2024-06-18T12:58:54.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.db_log_dir: 2024-06-18T12:58:54.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.wal_dir: 2024-06-18T12:58:54.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-18T12:58:54.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-18T12:58:54.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-18T12:58:54.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-18T12:58:54.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-18T12:58:54.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-18T12:58:54.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.advise_random_on_open: 1 2024-06-18T12:58:54.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.db_write_buffer_size: 0 2024-06-18T12:58:54.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.write_buffer_manager: 0x560a5fe981e0 2024-06-18T12:58:54.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-18T12:58:54.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-18T12:58:54.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-18T12:58:54.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.rate_limiter: (nil) 2024-06-18T12:58:54.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-18T12:58:54.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.wal_recovery_mode: 2 2024-06-18T12:58:54.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.enable_thread_tracking: 0 2024-06-18T12:58:54.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.enable_pipelined_write: 0 2024-06-18T12:58:54.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.unordered_write: 0 2024-06-18T12:58:54.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-18T12:58:54.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-18T12:58:54.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-18T12:58:54.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-18T12:58:54.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.row_cache: None 2024-06-18T12:58:54.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.wal_filter: None 2024-06-18T12:58:54.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-18T12:58:54.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.allow_ingest_behind: 0 2024-06-18T12:58:54.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.two_write_queues: 0 2024-06-18T12:58:54.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.manual_wal_flush: 0 2024-06-18T12:58:54.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.wal_compression: 0 2024-06-18T12:58:54.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.atomic_flush: 0 2024-06-18T12:58:54.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-18T12:58:54.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-18T12:58:54.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-18T12:58:54.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.log_readahead_size: 0 2024-06-18T12:58:54.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-18T12:58:54.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.best_efforts_recovery: 0 2024-06-18T12:58:54.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-18T12:58:54.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-18T12:58:54.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.allow_data_in_errors: 0 2024-06-18T12:58:54.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.db_host_id: __hostname__ 2024-06-18T12:58:54.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-18T12:58:54.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_background_jobs: 2 2024-06-18T12:58:54.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_background_compactions: -1 2024-06-18T12:58:54.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_subcompactions: 1 2024-06-18T12:58:54.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-18T12:58:54.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-18T12:58:54.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-18T12:58:54.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_total_wal_size: 0 2024-06-18T12:58:54.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_open_files: -1 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.bytes_per_sync: 0 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compaction_readahead_size: 0 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_background_flushes: -1 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Compression algorithms supported: 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: kZSTD supported: 0 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: kXpressCompression supported: 0 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: kBZip2Compression supported: 0 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-18T12:58:54.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: kLZ4Compression supported: 1 2024-06-18T12:58:54.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: kZlibCompression supported: 1 2024-06-18T12:58:54.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: kLZ4HCCompression supported: 1 2024-06-18T12:58:54.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: kSnappyCompression supported: 1 2024-06-18T12:58:54.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-18T12:58:54.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-18T12:58:54.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-06-18T12:58:54.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-18T12:58:54.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-18T12:58:54.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.merge_operator: 2024-06-18T12:58:54.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compaction_filter: None 2024-06-18T12:58:54.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compaction_filter_factory: None 2024-06-18T12:58:54.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.sst_partitioner_factory: None 2024-06-18T12:58:54.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-18T12:58:54.487 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-18T12:58:54.487 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560a5fe88d00) 2024-06-18T12:58:54.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: cache_index_and_filter_blocks: 1 2024-06-18T12:58:54.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-18T12:58:54.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-18T12:58:54.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: pin_top_level_index_and_filter: 1 2024-06-18T12:58:54.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_type: 0 2024-06-18T12:58:54.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: data_block_index_type: 0 2024-06-18T12:58:54.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_shortening: 1 2024-06-18T12:58:54.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-18T12:58:54.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: checksum: 4 2024-06-18T12:58:54.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: no_block_cache: 0 2024-06-18T12:58:54.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache: 0x560a5fe7f090 2024-06-18T12:58:54.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_name: BinnedLRUCache 2024-06-18T12:58:54.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_options: 2024-06-18T12:58:54.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: capacity : 536870912 2024-06-18T12:58:54.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: num_shard_bits : 4 2024-06-18T12:58:54.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: strict_capacity_limit : 0 2024-06-18T12:58:54.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: high_pri_pool_ratio: 0.000 2024-06-18T12:58:54.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_compressed: (nil) 2024-06-18T12:58:54.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: persistent_cache: (nil) 2024-06-18T12:58:54.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_size: 4096 2024-06-18T12:58:54.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_size_deviation: 10 2024-06-18T12:58:54.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_restart_interval: 16 2024-06-18T12:58:54.489 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_block_restart_interval: 1 2024-06-18T12:58:54.489 INFO:journalctl@ceph.mon.c.smithi167.stdout: metadata_block_size: 4096 2024-06-18T12:58:54.489 INFO:journalctl@ceph.mon.c.smithi167.stdout: partition_filters: 0 2024-06-18T12:58:54.489 INFO:journalctl@ceph.mon.c.smithi167.stdout: use_delta_encoding: 1 2024-06-18T12:58:54.489 INFO:journalctl@ceph.mon.c.smithi167.stdout: filter_policy: bloomfilter 2024-06-18T12:58:54.489 INFO:journalctl@ceph.mon.c.smithi167.stdout: whole_key_filtering: 1 2024-06-18T12:58:54.489 INFO:journalctl@ceph.mon.c.smithi167.stdout: verify_compression: 0 2024-06-18T12:58:54.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: read_amp_bytes_per_bit: 0 2024-06-18T12:58:54.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: format_version: 5 2024-06-18T12:58:54.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: enable_index_compression: 1 2024-06-18T12:58:54.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_align: 0 2024-06-18T12:58:54.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: max_auto_readahead_size: 262144 2024-06-18T12:58:54.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: prepopulate_block_cache: 0 2024-06-18T12:58:54.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: initial_auto_readahead_size: 8192 2024-06-18T12:58:54.491 INFO:journalctl@ceph.mon.c.smithi167.stdout: num_file_reads_for_auto_readahead: 2 2024-06-18T12:58:54.491 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T12:58:54.492 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.write_buffer_size: 33554432 2024-06-18T12:58:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_write_buffer_number: 2 2024-06-18T12:58:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compression: NoCompression 2024-06-18T12:58:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.bottommost_compression: Disabled 2024-06-18T12:58:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.prefix_extractor: nullptr 2024-06-18T12:58:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-18T12:58:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.num_levels: 7 2024-06-18T12:58:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-18T12:58:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-18T12:58:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-18T12:58:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-18T12:58:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-18T12:58:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-18T12:58:54.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-18T12:58:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-18T12:58:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-18T12:58:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-18T12:58:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-18T12:58:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-18T12:58:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-18T12:58:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compression_opts.level: 32767 2024-06-18T12:58:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compression_opts.strategy: 0 2024-06-18T12:58:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-18T12:58:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-18T12:58:54.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-18T12:58:54.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-18T12:58:54.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compression_opts.enabled: false 2024-06-18T12:58:54.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-18T12:58:54.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-18T12:58:54.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-18T12:58:54.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-18T12:58:54.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.target_file_size_base: 67108864 2024-06-18T12:58:54.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-18T12:58:54.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-18T12:58:54.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-18T12:58:54.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-18T12:58:54.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-18T12:58:54.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-18T12:58:54.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-18T12:58:54.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-18T12:58:54.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-18T12:58:54.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-18T12:58:54.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-18T12:58:54.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-18T12:58:54.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-18T12:58:54.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-18T12:58:54.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.arena_block_size: 1048576 2024-06-18T12:58:54.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-18T12:58:54.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-18T12:58:54.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.disable_auto_compactions: 0 2024-06-18T12:58:54.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-18T12:58:54.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-18T12:58:54.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-18T12:58:54.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-18T12:58:54.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-18T12:58:54.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-18T12:58:54.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-18T12:58:54.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-18T12:58:54.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-18T12:58:54.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-18T12:58:54.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.table_properties_collectors: 2024-06-18T12:58:54.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.inplace_update_support: 0 2024-06-18T12:58:54.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-18T12:58:54.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-18T12:58:54.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-18T12:58:54.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-18T12:58:54.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.bloom_locality: 0 2024-06-18T12:58:54.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.max_successive_merges: 0 2024-06-18T12:58:54.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-18T12:58:54.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.paranoid_file_checks: 0 2024-06-18T12:58:54.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.force_consistency_checks: 1 2024-06-18T12:58:54.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.report_bg_io_stats: 0 2024-06-18T12:58:54.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.ttl: 2592000 2024-06-18T12:58:54.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-18T12:58:54.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-18T12:58:54.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-18T12:58:54.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.enable_blob_files: false 2024-06-18T12:58:54.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.min_blob_size: 0 2024-06-18T12:58:54.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.blob_file_size: 268435456 2024-06-18T12:58:54.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-18T12:58:54.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-18T12:58:54.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-18T12:58:54.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-18T12:58:54.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-18T12:58:54.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.blob_file_starting_level: 0 2024-06-18T12:58:54.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-18T12:58:54.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-18T12:58:54.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-06-18T12:58:54.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f47da54d-b2ac-47b7-a93c-06175ed8019b 2024-06-18T12:58:54.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718715534148607, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-18T12:58:54.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-06-18T12:58:54.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718715534149018, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1718715534, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f47da54d-b2ac-47b7-a93c-06175ed8019b", "db_session_id": "TSIWY9H61FIJZG3IQDST", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-06-18T12:58:54.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718715534149109, "job": 1, "event": "recovery_finished"} 2024-06-18T12:58:54.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-06-18T12:58:54.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-18T12:58:54.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560a5feace00 2024-06-18T12:58:54.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: DB pointer 0x560a5ffa2000 2024-06-18T12:58:54.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-18T12:58:54.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-18T12:58:54.500 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** DB Stats ** 2024-06-18T12:58:54.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-18T12:58:54.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-18T12:58:54.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-18T12:58:54.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-18T12:58:54.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-18T12:58:54.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-18T12:58:54.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-18T12:58:54.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-06-18T12:58:54.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** Compaction Stats [default] ** 2024-06-18T12:58:54.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-18T12:58:54.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-18T12:58:54.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-18T12:58:54.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-18T12:58:54.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-18T12:58:54.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-06-18T12:58:54.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** Compaction Stats [default] ** 2024-06-18T12:58:54.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-18T12:58:54.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-18T12:58:54.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-18T12:58:54.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-06-18T12:58:54.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-18T12:58:54.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-06-18T12:58:54.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-18T12:58:54.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-18T12:58:54.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-18T12:58:54.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-18T12:58:54.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-18T12:58:54.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-18T12:58:54.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-18T12:58:54.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-18T12:58:54.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-18T12:58:54.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: Block cache BinnedLRUCache@0x560a5fe7f090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-18T12:58:54.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-06-18T12:58:54.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-06-18T12:58:54.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-18T12:58:54.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-06-18T12:58:54.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: using public_addrv [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] 2024-06-18T12:58:54.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: starting mon.c rank -1 at public addrs [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] at bind addrs [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:58:54.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c@-1(???) e0 preinit fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:58:54.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c@-1(synchronizing).mds e1 new map 2024-06-18T12:58:54.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c@-1(synchronizing).mds e1 print_map 2024-06-18T12:58:54.504 INFO:journalctl@ceph.mon.c.smithi167.stdout: e1 2024-06-18T12:58:54.504 INFO:journalctl@ceph.mon.c.smithi167.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-18T12:58:54.504 INFO:journalctl@ceph.mon.c.smithi167.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-18T12:58:54.504 INFO:journalctl@ceph.mon.c.smithi167.stdout: legacy client fscid: -1 2024-06-18T12:58:54.504 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-06-18T12:58:54.504 INFO:journalctl@ceph.mon.c.smithi167.stdout: No filesystems configured 2024-06-18T12:58:54.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-18T12:58:54.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-18T12:58:54.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-18T12:58:54.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-18T12:58:54.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-18T12:58:54.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: Updating smithi179:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T12:58:54.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-06-18T12:58:54.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: Updating smithi179:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-18T12:58:54.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2023705182' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-18T12:58:54.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:54.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:54.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:54.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2023705182' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-18T12:58:54.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: osdmap e4: 0 total, 0 up, 0 in 2024-06-18T12:58:54.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:54.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi167:172.21.15.167=a;smithi167:[v2:172.21.15.167:3301,v1:172.21.15.167:6790]=c;smithi179:172.21.15.179=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:58:54.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: Saving service mon spec with placement smithi167:172.21.15.167=a;smithi167:[v2:172.21.15.167:3301,v1:172.21.15.167:6790]=c;smithi179:172.21.15.179=b;count:3 2024-06-18T12:58:54.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:54.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:58:54.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:58:54.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:58:54.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:54.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-18T12:58:54.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:58:54.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: Deploying daemon mon.b on smithi179 2024-06-18T12:58:54.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:54.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:54.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:54.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T12:58:54.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.a calling monitor election 2024-06-18T12:58:54.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:54.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:54.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:54.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:54.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.b calling monitor election 2024-06-18T12:58:54.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:54.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:54.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:54.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:54.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-06-18T12:58:54.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: monmap epoch 2 2024-06-18T12:58:54.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:58:54.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: last_changed 2024-06-18T12:58:45.999439+0000 2024-06-18T12:58:54.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: created 2024-06-18T12:57:16.004836+0000 2024-06-18T12:58:54.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: min_mon_release 19 (squid) 2024-06-18T12:58:54.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: election_strategy: 1 2024-06-18T12:58:54.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: 0: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.a 2024-06-18T12:58:54.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.b 2024-06-18T12:58:54.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: fsmap 2024-06-18T12:58:54.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: osdmap e4: 0 total, 0 up, 0 in 2024-06-18T12:58:54.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mgrmap e13: y(active, since 37s) 2024-06-18T12:58:54.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: overall HEALTH_OK 2024-06-18T12:58:54.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:54.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:54.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:54.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-18T12:58:54.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:58:54.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: Deploying daemon mon.c on smithi167 2024-06-18T12:58:54.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:54.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:54.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.179:0/3577703307' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-18T12:58:54.510 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:54.510 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 ceph-mon[37701]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-18T12:58:54.510 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 bash[37680]: de3cc70740a845bc58ce314fbe054ccc5071896eddc5c83a101ebf1acfb521e9 2024-06-18T12:58:54.510 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:54 smithi167 systemd[1]: Started Ceph mon.c for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T12:58:59.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.500+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/110048873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe690003530 msgr2=0x7fe6900059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:59.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.500+0000 7fe6a6f1f640 1 --2- 172.21.15.179:0/110048873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe690003530 0x7fe6900059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe6a01011a0 tx=0x7fe69402c3f0 comp rx=0 tx=0).stop 2024-06-18T12:58:59.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.500+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/110048873 shutdown_connections 2024-06-18T12:58:59.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.500+0000 7fe6a6f1f640 1 --2- 172.21.15.179:0/110048873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe690003530 0x7fe6900059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:59.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.500+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/110048873 >> 172.21.15.179:0/110048873 conn(0x7fe6a00fb9d0 msgr2=0x7fe6a00fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:59.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.500+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/110048873 shutdown_connections 2024-06-18T12:58:59.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.500+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/110048873 wait complete. 2024-06-18T12:58:59.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.501+0000 7fe6a6f1f640 1 Processor -- start 2024-06-18T12:58:59.501 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T12:58:59.501 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: mon.a calling monitor election 2024-06-18T12:58:59.502 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:59.505 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.505 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: mon.b calling monitor election 2024-06-18T12:58:59.505 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.505 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:59.505 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.505 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: mon.c calling monitor election 2024-06-18T12:58:59.505 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.505 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:59.506 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.506 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.506 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-06-18T12:58:59.506 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: monmap epoch 3 2024-06-18T12:58:59.506 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:58:59.506 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: last_changed 2024-06-18T12:58:54.180742+0000 2024-06-18T12:58:59.506 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: created 2024-06-18T12:57:16.004836+0000 2024-06-18T12:58:59.506 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: min_mon_release 19 (squid) 2024-06-18T12:58:59.506 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: election_strategy: 1 2024-06-18T12:58:59.506 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: 0: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.a 2024-06-18T12:58:59.506 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.b 2024-06-18T12:58:59.507 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: 2: [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] mon.c 2024-06-18T12:58:59.507 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: fsmap 2024-06-18T12:58:59.507 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: osdmap e4: 0 total, 0 up, 0 in 2024-06-18T12:58:59.507 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: mgrmap e13: y(active, since 45s) 2024-06-18T12:58:59.507 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: overall HEALTH_OK 2024-06-18T12:58:59.507 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:59.507 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:59.507 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:59.507 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:58:59 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:59.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.501+0000 7fe6a6f1f640 1 -- start start 2024-06-18T12:58:59.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.501+0000 7fe6a6f1f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe690003530 0x7fe6a0196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:59.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.501+0000 7fe6a6f1f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a01973f0 0x7fe6a0197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:59.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.501+0000 7fe6a6f1f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6a019ff20 0x7fe6a01a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:59.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.501+0000 7fe6a6f1f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01072a0 con 0x7fe6a01973f0 2024-06-18T12:58:59.510 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.501+0000 7fe6a6f1f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0107120 con 0x7fe6a019ff20 2024-06-18T12:58:59.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.501+0000 7fe6a6f1f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a0107420 con 0x7fe690003530 2024-06-18T12:58:59.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe69ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a01973f0 0x7fe6a0197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:59.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe6a4cbb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe690003530 0x7fe6a0196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:59.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe6a54bc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6a019ff20 0x7fe6a01a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:59.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe6a4cbb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe690003530 0x7fe6a0196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.179:45266/0 (socket says 172.21.15.179:45266) 2024-06-18T12:58:59.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe6a4cbb640 1 -- 172.21.15.179:0/778116409 learned_addr learned my addr 172.21.15.179:0/778116409 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T12:58:59.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe6a54bc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6a019ff20 0x7fe6a01a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:45754/0 (socket says 172.21.15.179:45754) 2024-06-18T12:58:59.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe6a4cbb640 1 -- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe690003530 msgr2=0x7fe6a0196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-18T12:58:59.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe6a4cbb640 1 -- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe690003530 msgr2=0x7fe6a0196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-18T12:58:59.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe6a4cbb640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe690003530 0x7fe6a0196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-18T12:58:59.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe6a4cbb640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe690003530 0x7fe6a0196eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-18T12:58:59.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe6a54bc640 1 -- 172.21.15.179:0/778116409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6a019ff20 msgr2=0x7fe6a01a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-06-18T12:58:59.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe6a54bc640 1 -- 172.21.15.179:0/778116409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6a019ff20 msgr2=0x7fe6a01a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-18T12:58:59.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe6a54bc640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6a019ff20 0x7fe6a01a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-18T12:58:59.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe6a54bc640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6a019ff20 0x7fe6a01a2310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-18T12:58:59.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe69ffff640 1 -- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe690003530 msgr2=0x7fe6a0196eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-18T12:58:59.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe69ffff640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe690003530 0x7fe6a0196eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:59.512 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe69ffff640 1 -- 172.21.15.179:0/778116409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6a019ff20 msgr2=0x7fe6a01a2310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-18T12:58:59.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe69ffff640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6a019ff20 0x7fe6a01a2310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:59.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe69ffff640 1 -- 172.21.15.179:0/778116409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a01a2ab0 con 0x7fe6a01973f0 2024-06-18T12:58:59.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.502+0000 7fe69ffff640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a01973f0 0x7fe6a0197850 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00b450 tx=0x7fe68c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:59.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.503+0000 7fe69dffb640 1 -- 172.21.15.179:0/778116409 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c004230 con 0x7fe6a01973f0 2024-06-18T12:58:59.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.503+0000 7fe69dffb640 1 -- 172.21.15.179:0/778116409 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fe68c0043d0 con 0x7fe6a01973f0 2024-06-18T12:58:59.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.503+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/778116409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a01a2da0 con 0x7fe6a01973f0 2024-06-18T12:58:59.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.503+0000 7fe69dffb640 1 -- 172.21.15.179:0/778116409 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c003af0 con 0x7fe6a01973f0 2024-06-18T12:58:59.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.503+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/778116409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a01a3300 con 0x7fe6a01973f0 2024-06-18T12:58:59.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.504+0000 7fe69dffb640 1 -- 172.21.15.179:0/778116409 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fe68c00f050 con 0x7fe6a01973f0 2024-06-18T12:58:59.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.504+0000 7fe69dffb640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe6840394f0 0x7fe68403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:58:59.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.504+0000 7fe69dffb640 1 -- 172.21.15.179:0/778116409 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe68c04bcd0 con 0x7fe6a01973f0 2024-06-18T12:58:59.513 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.504+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/778116409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a0108200 con 0x7fe6a01973f0 2024-06-18T12:58:59.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.504+0000 7fe6a4cbb640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe6840394f0 0x7fe68403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:58:59.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.505+0000 7fe6a4cbb640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe6840394f0 0x7fe68403b9b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe694005fd0 tx=0x7fe694005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:58:59.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.510+0000 7fe69dffb640 1 -- 172.21.15.179:0/778116409 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fe68c00d040 con 0x7fe6a01973f0 2024-06-18T12:58:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T12:58:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: mon.a calling monitor election 2024-06-18T12:58:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: mon.b calling monitor election 2024-06-18T12:58:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: mon.c calling monitor election 2024-06-18T12:58:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-06-18T12:58:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: monmap epoch 3 2024-06-18T12:58:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:58:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: last_changed 2024-06-18T12:58:54.180742+0000 2024-06-18T12:58:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: created 2024-06-18T12:57:16.004836+0000 2024-06-18T12:58:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: min_mon_release 19 (squid) 2024-06-18T12:58:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: election_strategy: 1 2024-06-18T12:58:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: 0: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.a 2024-06-18T12:58:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.b 2024-06-18T12:58:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: 2: [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] mon.c 2024-06-18T12:58:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: fsmap 2024-06-18T12:58:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: osdmap e4: 0 total, 0 up, 0 in 2024-06-18T12:58:59.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: mgrmap e13: y(active, since 45s) 2024-06-18T12:58:59.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: overall HEALTH_OK 2024-06-18T12:58:59.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:59.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:59.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:59.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:59.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T12:58:59.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: mon.a calling monitor election 2024-06-18T12:58:59.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T12:58:59.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: mon.b calling monitor election 2024-06-18T12:58:59.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:59.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: mon.c calling monitor election 2024-06-18T12:58:59.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:58:59.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:58:59.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-06-18T12:58:59.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: monmap epoch 3 2024-06-18T12:58:59.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:58:59.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: last_changed 2024-06-18T12:58:54.180742+0000 2024-06-18T12:58:59.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: created 2024-06-18T12:57:16.004836+0000 2024-06-18T12:58:59.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: min_mon_release 19 (squid) 2024-06-18T12:58:59.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: election_strategy: 1 2024-06-18T12:58:59.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: 0: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.a 2024-06-18T12:58:59.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.b 2024-06-18T12:58:59.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: 2: [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] mon.c 2024-06-18T12:58:59.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: fsmap 2024-06-18T12:58:59.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: osdmap e4: 0 total, 0 up, 0 in 2024-06-18T12:58:59.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: mgrmap e13: y(active, since 45s) 2024-06-18T12:58:59.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: overall HEALTH_OK 2024-06-18T12:58:59.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:59.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:59.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:59.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:58:59 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:58:59.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.772+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/778116409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe6a006a1d0 con 0x7fe6a01973f0 2024-06-18T12:58:59.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.772+0000 7fe69dffb640 1 -- 172.21.15.179:0/778116409 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7fe6a006a1d0 con 0x7fe6a01973f0 2024-06-18T12:58:59.772 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-18T12:58:59.772 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":3,"fsid":"3434f1e2-2d72-11ef-bc9d-c7b262605968","modified":"2024-06-18T12:58:54.180742Z","created":"2024-06-18T12:57:16.004836Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3301","nonce":0},{"type":"v1","addr":"172.21.15.167:6790","nonce":0}]},"addr":"172.21.15.167:6790/0","public_addr":"172.21.15.167:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-06-18T12:58:59.772 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 3 2024-06-18T12:58:59.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.774+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe6840394f0 msgr2=0x7fe68403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:59.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.774+0000 7fe6a6f1f640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe6840394f0 0x7fe68403b9b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe694005fd0 tx=0x7fe694005f60 comp rx=0 tx=0).stop 2024-06-18T12:58:59.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.774+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a01973f0 msgr2=0x7fe6a0197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:58:59.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.774+0000 7fe6a6f1f640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a01973f0 0x7fe6a0197850 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00b450 tx=0x7fe68c00b910 comp rx=0 tx=0).stop 2024-06-18T12:58:59.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.774+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/778116409 shutdown_connections 2024-06-18T12:58:59.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.774+0000 7fe6a6f1f640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe6840394f0 0x7fe68403b9b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:59.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.774+0000 7fe6a6f1f640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6a019ff20 0x7fe6a01a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:59.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.774+0000 7fe6a6f1f640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a01973f0 0x7fe6a0197850 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:59.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.774+0000 7fe6a6f1f640 1 --2- 172.21.15.179:0/778116409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe690003530 0x7fe6a0196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:58:59.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.774+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/778116409 >> 172.21.15.179:0/778116409 conn(0x7fe6a00fb9d0 msgr2=0x7fe6a0103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:58:59.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.775+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/778116409 shutdown_connections 2024-06-18T12:58:59.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:58:59.775+0000 7fe6a6f1f640 1 -- 172.21.15.179:0/778116409 wait complete. 2024-06-18T12:59:00.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:00 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:00.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:00 smithi179 ceph-mon[33312]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:00.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.179:0/778116409' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-18T12:59:00.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:00 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:59:00.433 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-18T12:59:00.433 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph config generate-minimal-conf 2024-06-18T12:59:00.666 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T12:59:00.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:00 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:00.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:00 smithi167 ceph-mon[37701]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.179:0/778116409' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-18T12:59:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:00 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:59:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:00 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:00 smithi167 ceph-mon[29496]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.179:0/778116409' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-18T12:59:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:00 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T12:59:01.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:01 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:01.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:01 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:01.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:01 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:01.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:01 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:01.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:01 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:01.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:01 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:01.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:01 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:01.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:01 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:01 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:01 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:01 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:01 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:01 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:01 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:01 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:01 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:01 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:01 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:02.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: Updating smithi167:/etc/ceph/ceph.conf 2024-06-18T12:59:02.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-18T12:59:02.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: Updating smithi167:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T12:59:02.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: Updating smithi179:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T12:59:02.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: Reconfiguring mon.a (unknown last config time)... 2024-06-18T12:59:02.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-18T12:59:02.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-18T12:59:02.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:02.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: Reconfiguring daemon mon.a on smithi167 2024-06-18T12:59:02.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[29496]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:02.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: Updating smithi167:/etc/ceph/ceph.conf 2024-06-18T12:59:02.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: Updating smithi167:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: Updating smithi179:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: Reconfiguring mon.a (unknown last config time)... 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-18T12:59:02.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:02.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: Reconfiguring daemon mon.a on smithi167 2024-06-18T12:59:02.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:02 smithi167 ceph-mon[37701]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:02.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: Updating smithi167:/etc/ceph/ceph.conf 2024-06-18T12:59:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-18T12:59:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: Updating smithi167:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T12:59:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: Updating smithi179:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T12:59:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:02.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: Reconfiguring mon.a (unknown last config time)... 2024-06-18T12:59:02.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-18T12:59:02.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-18T12:59:02.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:02.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: Reconfiguring daemon mon.a on smithi167 2024-06-18T12:59:02.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:02 smithi179 ceph-mon[33312]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:03.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.188+0000 7f85c9c4e640 1 -- 172.21.15.167:0/176979403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85c41027d0 msgr2=0x7f85c410d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:03.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.188+0000 7f85c9c4e640 1 --2- 172.21.15.167:0/176979403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85c41027d0 0x7f85c410d1f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009cd0 tx=0x7f85bc02a8e0 comp rx=0 tx=0).stop 2024-06-18T12:59:03.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.188+0000 7f85c9c4e640 1 -- 172.21.15.167:0/176979403 shutdown_connections 2024-06-18T12:59:03.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.188+0000 7f85c9c4e640 1 --2- 172.21.15.167:0/176979403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85c41027d0 0x7f85c410d1f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:03.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.188+0000 7f85c9c4e640 1 --2- 172.21.15.167:0/176979403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85c4101e50 0x7f85c4102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:03.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.188+0000 7f85c9c4e640 1 --2- 172.21.15.167:0/176979403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85c4100560 0x7f85c4100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:03.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.188+0000 7f85c9c4e640 1 -- 172.21.15.167:0/176979403 >> 172.21.15.167:0/176979403 conn(0x7f85c40fbf80 msgr2=0x7f85c40fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:59:03.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.188+0000 7f85c9c4e640 1 -- 172.21.15.167:0/176979403 shutdown_connections 2024-06-18T12:59:03.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.188+0000 7f85c9c4e640 1 -- 172.21.15.167:0/176979403 wait complete. 2024-06-18T12:59:03.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c9c4e640 1 Processor -- start 2024-06-18T12:59:03.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c9c4e640 1 -- start start 2024-06-18T12:59:03.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c9c4e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85c4100560 0x7f85c41971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:03.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c9c4e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85c4101e50 0x7f85c4197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:03.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c9c4e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85c41027d0 0x7f85c419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:03.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c9c4e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f85c410fbd0 con 0x7f85c41027d0 2024-06-18T12:59:03.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c9c4e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f85c410fa50 con 0x7f85c4101e50 2024-06-18T12:59:03.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c9c4e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f85c410fd50 con 0x7f85c4100560 2024-06-18T12:59:03.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c37fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85c4100560 0x7f85c41971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:03.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85c41027d0 0x7f85c419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:03.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c37fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85c4100560 0x7f85c41971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:52010/0 (socket says 172.21.15.167:52010) 2024-06-18T12:59:03.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c37fe640 1 -- 172.21.15.167:0/675359960 learned_addr learned my addr 172.21.15.167:0/675359960 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:59:03.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85baffd640 1 --2- 172.21.15.167:0/675359960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85c4101e50 0x7f85c4197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:03.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c37fe640 1 -- 172.21.15.167:0/675359960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85c4101e50 msgr2=0x7f85c4197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:03.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c37fe640 1 --2- 172.21.15.167:0/675359960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85c4101e50 0x7f85c4197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:03.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c37fe640 1 -- 172.21.15.167:0/675359960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85c41027d0 msgr2=0x7f85c419bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:03.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c37fe640 1 --2- 172.21.15.167:0/675359960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85c41027d0 0x7f85c419bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:03.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85c37fe640 1 -- 172.21.15.167:0/675359960 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c419c190 con 0x7f85c4100560 2024-06-18T12:59:03.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.189+0000 7f85baffd640 1 --2- 172.21.15.167:0/675359960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85c4101e50 0x7f85c4197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T12:59:03.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.190+0000 7f85c37fe640 1 --2- 172.21.15.167:0/675359960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85c4100560 0x7f85c41971e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f85b400b4a0 tx=0x7f85b400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:59:03.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.190+0000 7f85c17fa640 1 -- 172.21.15.167:0/675359960 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4014070 con 0x7f85c4100560 2024-06-18T12:59:03.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.190+0000 7f85c17fa640 1 -- 172.21.15.167:0/675359960 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f85b4010a90 con 0x7f85c4100560 2024-06-18T12:59:03.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.190+0000 7f85c9c4e640 1 -- 172.21.15.167:0/675359960 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c419c480 con 0x7f85c4100560 2024-06-18T12:59:03.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.190+0000 7f85c17fa640 1 -- 172.21.15.167:0/675359960 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4017b40 con 0x7f85c4100560 2024-06-18T12:59:03.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.190+0000 7f85c9c4e640 1 -- 172.21.15.167:0/675359960 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c40fc360 con 0x7f85c4100560 2024-06-18T12:59:03.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.191+0000 7f85c17fa640 1 -- 172.21.15.167:0/675359960 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f85b4017070 con 0x7f85c4100560 2024-06-18T12:59:03.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.191+0000 7f85c9c4e640 1 -- 172.21.15.167:0/675359960 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85c4100940 con 0x7f85c4100560 2024-06-18T12:59:03.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.191+0000 7f85c17fa640 1 --2- 172.21.15.167:0/675359960 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f85b0039290 0x7f85b003b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:03.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.191+0000 7f85c17fa640 1 -- 172.21.15.167:0/675359960 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f85b404ba90 con 0x7f85c4100560 2024-06-18T12:59:03.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.191+0000 7f85baffd640 1 --2- 172.21.15.167:0/675359960 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f85b0039290 0x7f85b003b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:03.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.191+0000 7f85baffd640 1 --2- 172.21.15.167:0/675359960 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f85b0039290 0x7f85b003b750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f85c4110770 tx=0x7f85ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:59:03.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.196+0000 7f85c17fa640 1 -- 172.21.15.167:0/675359960 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f85b4014210 con 0x7f85c4100560 2024-06-18T12:59:03.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.382+0000 7f85c9c4e640 1 -- 172.21.15.167:0/675359960 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f85c40fc620 con 0x7f85c4100560 2024-06-18T12:59:03.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.383+0000 7f85c17fa640 1 -- 172.21.15.167:0/675359960 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f85b4021e30 con 0x7f85c4100560 2024-06-18T12:59:03.383 INFO:teuthology.orchestra.run.smithi167.stdout:# minimal ceph.conf for 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:59:03.383 INFO:teuthology.orchestra.run.smithi167.stdout:[global] 2024-06-18T12:59:03.383 INFO:teuthology.orchestra.run.smithi167.stdout: fsid = 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T12:59:03.383 INFO:teuthology.orchestra.run.smithi167.stdout: mon_host = [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] 2024-06-18T12:59:03.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.385+0000 7f85c9c4e640 1 -- 172.21.15.167:0/675359960 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f85b0039290 msgr2=0x7f85b003b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:03.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.385+0000 7f85c9c4e640 1 --2- 172.21.15.167:0/675359960 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f85b0039290 0x7f85b003b750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f85c4110770 tx=0x7f85ac005f90 comp rx=0 tx=0).stop 2024-06-18T12:59:03.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.385+0000 7f85c9c4e640 1 -- 172.21.15.167:0/675359960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85c4100560 msgr2=0x7f85c41971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:03.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.385+0000 7f85c9c4e640 1 --2- 172.21.15.167:0/675359960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85c4100560 0x7f85c41971e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f85b400b4a0 tx=0x7f85b400b960 comp rx=0 tx=0).stop 2024-06-18T12:59:03.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.385+0000 7f85c9c4e640 1 -- 172.21.15.167:0/675359960 shutdown_connections 2024-06-18T12:59:03.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.385+0000 7f85c9c4e640 1 --2- 172.21.15.167:0/675359960 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f85b0039290 0x7f85b003b750 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:03.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.385+0000 7f85c9c4e640 1 --2- 172.21.15.167:0/675359960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85c41027d0 0x7f85c419bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:03.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.385+0000 7f85c9c4e640 1 --2- 172.21.15.167:0/675359960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85c4101e50 0x7f85c4197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:03.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.385+0000 7f85c9c4e640 1 --2- 172.21.15.167:0/675359960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85c4100560 0x7f85c41971e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:03.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.385+0000 7f85c9c4e640 1 -- 172.21.15.167:0/675359960 >> 172.21.15.167:0/675359960 conn(0x7f85c40fbf80 msgr2=0x7f85c4104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:59:03.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.385+0000 7f85c9c4e640 1 -- 172.21.15.167:0/675359960 shutdown_connections 2024-06-18T12:59:03.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:03.385+0000 7f85c9c4e640 1 -- 172.21.15.167:0/675359960 wait complete. 2024-06-18T12:59:04.026 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-18T12:59:04.026 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T12:59:04.026 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.conf 2024-06-18T12:59:04.049 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:04.049 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:04.049 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[29496]: Reconfiguring mon.c (monmap changed)... 2024-06-18T12:59:04.049 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-18T12:59:04.049 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-18T12:59:04.050 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:04.050 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[29496]: Reconfiguring daemon mon.c on smithi167 2024-06-18T12:59:04.050 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/675359960' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:04.050 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[29496]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:04.050 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:04.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:04.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:04.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[37701]: Reconfiguring mon.c (monmap changed)... 2024-06-18T12:59:04.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-18T12:59:04.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-18T12:59:04.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:04.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[37701]: Reconfiguring daemon mon.c on smithi167 2024-06-18T12:59:04.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/675359960' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:04.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[37701]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:04.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:03 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:04.053 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T12:59:04.053 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-18T12:59:04.118 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-18T12:59:04.118 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.conf 2024-06-18T12:59:04.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:03 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:04.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:03 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:04.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:03 smithi179 ceph-mon[33312]: Reconfiguring mon.c (monmap changed)... 2024-06-18T12:59:04.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:03 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-18T12:59:04.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:03 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-18T12:59:04.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:03 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:04.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:03 smithi179 ceph-mon[33312]: Reconfiguring daemon mon.c on smithi167 2024-06-18T12:59:04.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:03 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/675359960' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:04.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:03 smithi179 ceph-mon[33312]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:04.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:03 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:04.150 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-18T12:59:04.150 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-18T12:59:04.220 INFO:tasks.cephadm:Adding mgr.y on smithi167 2024-06-18T12:59:04.220 INFO:tasks.cephadm:Adding mgr.x on smithi179 2024-06-18T12:59:04.220 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch apply mgr '2;smithi167=y;smithi179=x' 2024-06-18T12:59:04.494 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T12:59:06.133 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:05 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:06.134 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:05 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:06.134 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:05 smithi179 ceph-mon[33312]: Reconfiguring mon.b (monmap changed)... 2024-06-18T12:59:06.134 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:05 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-18T12:59:06.134 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:05 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-18T12:59:06.134 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:05 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:06.134 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:05 smithi179 ceph-mon[33312]: Reconfiguring daemon mon.b on smithi179 2024-06-18T12:59:06.134 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:05 smithi179 ceph-mon[33312]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:06.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.133+0000 7f4087be8640 1 -- 172.21.15.179:0/1206211253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40800ff720 msgr2=0x7f40800ffb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:06.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.133+0000 7f4087be8640 1 --2- 172.21.15.179:0/1206211253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40800ff720 0x7f40800ffb20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4078009880 tx=0x7f407802a5d0 comp rx=0 tx=0).stop 2024-06-18T12:59:06.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.133+0000 7f4087be8640 1 -- 172.21.15.179:0/1206211253 shutdown_connections 2024-06-18T12:59:06.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.133+0000 7f4087be8640 1 --2- 172.21.15.179:0/1206211253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40800ff720 0x7f40800ffb20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:06.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.133+0000 7f4087be8640 1 -- 172.21.15.179:0/1206211253 >> 172.21.15.179:0/1206211253 conn(0x7f40800fb550 msgr2=0x7f40800fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:59:06.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.133+0000 7f4087be8640 1 -- 172.21.15.179:0/1206211253 shutdown_connections 2024-06-18T12:59:06.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.133+0000 7f4087be8640 1 -- 172.21.15.179:0/1206211253 wait complete. 2024-06-18T12:59:06.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.134+0000 7f4087be8640 1 Processor -- start 2024-06-18T12:59:06.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.134+0000 7f4087be8640 1 -- start start 2024-06-18T12:59:06.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.134+0000 7f4087be8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40800ff720 0x7f4080197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:06.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.134+0000 7f4087be8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4080198410 0x7f408019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:06.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.134+0000 7f4087be8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4080198890 0x7f408019fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:06.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.134+0000 7f4087be8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40801085d0 con 0x7f4080198410 2024-06-18T12:59:06.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.134+0000 7f4087be8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4080108450 con 0x7f4080198890 2024-06-18T12:59:06.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.134+0000 7f4087be8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4080108750 con 0x7f40800ff720 2024-06-18T12:59:06.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.134+0000 7f4086185640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4080198890 0x7f408019fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:06.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.134+0000 7f4086185640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4080198890 0x7f408019fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53668/0 (socket says 172.21.15.179:53668) 2024-06-18T12:59:06.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.134+0000 7f4086185640 1 -- 172.21.15.179:0/4133278550 learned_addr learned my addr 172.21.15.179:0/4133278550 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T12:59:06.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.135+0000 7f4085183640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4080198410 0x7f408019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:06.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.135+0000 7f4085984640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40800ff720 0x7f4080197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:06.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.135+0000 7f4086185640 1 -- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40800ff720 msgr2=0x7f4080197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:06.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.135+0000 7f4086185640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40800ff720 0x7f4080197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:06.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.135+0000 7f4086185640 1 -- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4080198410 msgr2=0x7f408019d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:06.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.135+0000 7f4086185640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4080198410 0x7f408019d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:06.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.135+0000 7f4086185640 1 -- 172.21.15.179:0/4133278550 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40801a02d0 con 0x7f4080198890 2024-06-18T12:59:06.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.135+0000 7f4085984640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40800ff720 0x7f4080197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T12:59:06.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.135+0000 7f4086185640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4080198890 0x7f408019fa10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f407000ba50 tx=0x7f407000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:59:06.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.135+0000 7f407effd640 1 -- 172.21.15.179:0/4133278550 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070019070 con 0x7f4080198890 2024-06-18T12:59:06.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.135+0000 7f4087be8640 1 -- 172.21.15.179:0/4133278550 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40801a05c0 con 0x7f4080198890 2024-06-18T12:59:06.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.135+0000 7f407effd640 1 -- 172.21.15.179:0/4133278550 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f407000cbe0 con 0x7f4080198890 2024-06-18T12:59:06.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.135+0000 7f407effd640 1 -- 172.21.15.179:0/4133278550 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40700075b0 con 0x7f4080198890 2024-06-18T12:59:06.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.135+0000 7f4087be8640 1 -- 172.21.15.179:0/4133278550 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40801a0b80 con 0x7f4080198890 2024-06-18T12:59:06.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.136+0000 7f407effd640 1 -- 172.21.15.179:0/4133278550 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f407001c070 con 0x7f4080198890 2024-06-18T12:59:06.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.136+0000 7f407effd640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f40680394f0 0x7f406803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:06.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.136+0000 7f407effd640 1 -- 172.21.15.179:0/4133278550 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f407004bae0 con 0x7f4080198890 2024-06-18T12:59:06.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.136+0000 7f4087be8640 1 -- 172.21.15.179:0/4133278550 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40800ffba0 con 0x7f4080198890 2024-06-18T12:59:06.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.137+0000 7f4085984640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f40680394f0 0x7f406803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:06.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.137+0000 7f4085984640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f40680394f0 0x7f406803b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f40780096c0 tx=0x7f40780058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:59:06.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.141+0000 7f407effd640 1 -- 172.21.15.179:0/4133278550 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f40700133a0 con 0x7f4080198890 2024-06-18T12:59:06.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:06.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:06.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[29496]: Reconfiguring mon.b (monmap changed)... 2024-06-18T12:59:06.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-18T12:59:06.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-18T12:59:06.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:06.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[29496]: Reconfiguring daemon mon.b on smithi179 2024-06-18T12:59:06.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[29496]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:06.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:06.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:06.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[37701]: Reconfiguring mon.b (monmap changed)... 2024-06-18T12:59:06.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-18T12:59:06.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-18T12:59:06.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:06.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[37701]: Reconfiguring daemon mon.b on smithi179 2024-06-18T12:59:06.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:05 smithi167 ceph-mon[37701]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:06.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.335+0000 7f4087be8640 1 -- 172.21.15.179:0/4133278550 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi167=y;smithi179=x", "target": ["mon-mgr", ""]}) -- 0x7f408002d610 con 0x7f40680394f0 2024-06-18T12:59:06.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.342+0000 7f407effd640 1 -- 172.21.15.179:0/4133278550 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f408002d610 con 0x7f40680394f0 2024-06-18T12:59:06.341 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mgr update... 2024-06-18T12:59:06.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.344+0000 7f4087be8640 1 -- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f40680394f0 msgr2=0x7f406803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:06.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.344+0000 7f4087be8640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f40680394f0 0x7f406803b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f40780096c0 tx=0x7f40780058b0 comp rx=0 tx=0).stop 2024-06-18T12:59:06.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.344+0000 7f4087be8640 1 -- 172.21.15.179:0/4133278550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4080198890 msgr2=0x7f408019fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:06.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.344+0000 7f4087be8640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4080198890 0x7f408019fa10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f407000ba50 tx=0x7f407000bf10 comp rx=0 tx=0).stop 2024-06-18T12:59:06.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.344+0000 7f4087be8640 1 -- 172.21.15.179:0/4133278550 shutdown_connections 2024-06-18T12:59:06.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.344+0000 7f4087be8640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f40680394f0 0x7f406803b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:06.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.344+0000 7f4087be8640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4080198890 0x7f408019fa10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:06.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.344+0000 7f4087be8640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4080198410 0x7f408019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:06.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.344+0000 7f4087be8640 1 --2- 172.21.15.179:0/4133278550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40800ff720 0x7f4080197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:06.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.344+0000 7f4087be8640 1 -- 172.21.15.179:0/4133278550 >> 172.21.15.179:0/4133278550 conn(0x7f40800fb550 msgr2=0x7f40800fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:59:06.344 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.344+0000 7f4087be8640 1 -- 172.21.15.179:0/4133278550 shutdown_connections 2024-06-18T12:59:06.345 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T12:59:06.344+0000 7f4087be8640 1 -- 172.21.15.179:0/4133278550 wait complete. 2024-06-18T12:59:07.079 DEBUG:teuthology.orchestra.run.smithi179:mgr.x> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mgr.x.service 2024-06-18T12:59:07.081 INFO:tasks.cephadm:Deploying OSDs... 2024-06-18T12:59:07.082 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T12:59:07.082 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-06-18T12:59:07.102 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-18T12:59:07.102 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-06-18T12:59:07.161 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-18T12:59:07.161 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-18T12:59:07.161 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-06-18T12:59:07.161 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-18T12:59:07.161 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-06-18T12:59:07.162 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-06-18 12:58:53.626261762 +0000 2024-06-18T12:59:07.162 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-06-18 12:53:22.282104382 +0000 2024-06-18T12:59:07.162 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-06-18 12:53:22.282104382 +0000 2024-06-18T12:59:07.162 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-06-18 12:53:22.282104382 +0000 2024-06-18T12:59:07.162 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-18T12:59:07.229 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-06-18T12:59:07.230 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-06-18T12:59:07.230 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000235947 s, 2.2 MB/s 2024-06-18T12:59:07.231 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-18T12:59:07.290 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-06-18T12:59:07.347 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-18T12:59:07.348 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-18T12:59:07.348 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-06-18T12:59:07.348 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-18T12:59:07.348 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-06-18T12:59:07.348 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-06-18 12:58:53.626261762 +0000 2024-06-18T12:59:07.348 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-06-18 12:53:22.620098498 +0000 2024-06-18T12:59:07.348 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-06-18 12:53:22.620098498 +0000 2024-06-18T12:59:07.348 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-06-18 12:53:22.620098498 +0000 2024-06-18T12:59:07.348 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-18T12:59:07.415 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-06-18T12:59:07.415 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-06-18T12:59:07.415 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000198484 s, 2.6 MB/s 2024-06-18T12:59:07.416 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-18T12:59:07.474 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-06-18T12:59:07.531 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-18T12:59:07.531 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-18T12:59:07.531 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-06-18T12:59:07.532 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-18T12:59:07.532 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-06-18T12:59:07.532 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-06-18 12:58:53.627261745 +0000 2024-06-18T12:59:07.532 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-06-18 12:53:22.952092719 +0000 2024-06-18T12:59:07.532 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-06-18 12:53:22.952092719 +0000 2024-06-18T12:59:07.532 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-06-18 12:53:22.952092719 +0000 2024-06-18T12:59:07.532 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-18T12:59:07.593 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:07 smithi167 ceph-mon[29496]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi167=y;smithi179=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:59:07.593 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:07 smithi167 ceph-mon[29496]: Saving service mgr spec with placement smithi167=y;smithi179=x;count:2 2024-06-18T12:59:07.593 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:07 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:07.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:07 smithi167 ceph-mon[37701]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi167=y;smithi179=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:59:07.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:07 smithi167 ceph-mon[37701]: Saving service mgr spec with placement smithi167=y;smithi179=x;count:2 2024-06-18T12:59:07.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:07 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:07.596 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-06-18T12:59:07.596 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-06-18T12:59:07.596 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000198005 s, 2.6 MB/s 2024-06-18T12:59:07.597 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-18T12:59:07.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:07 smithi179 ceph-mon[33312]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi167=y;smithi179=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:59:07.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:07 smithi179 ceph-mon[33312]: Saving service mgr spec with placement smithi167=y;smithi179=x;count:2 2024-06-18T12:59:07.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:07 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:07.654 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-06-18T12:59:07.710 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-18T12:59:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-18T12:59:07.711 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-18T12:59:07.711 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-18T12:59:07.711 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-06-18T12:59:07.711 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-06-18 12:58:53.627261745 +0000 2024-06-18T12:59:07.711 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-06-18 12:53:23.279087026 +0000 2024-06-18T12:59:07.711 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-06-18 12:53:23.279087026 +0000 2024-06-18T12:59:07.711 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-06-18 12:53:23.279087026 +0000 2024-06-18T12:59:07.711 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-18T12:59:07.775 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-06-18T12:59:07.775 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-06-18T12:59:07.775 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000218522 s, 2.3 MB/s 2024-06-18T12:59:07.777 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-18T12:59:07.833 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-18T12:59:07.833 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/scratch_devs of=/dev/stdout 2024-06-18T12:59:07.849 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-18T12:59:07.849 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_1 2024-06-18T12:59:07.904 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-18T12:59:07.904 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-18T12:59:07.904 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-06-18T12:59:07.904 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-18T12:59:07.905 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-06-18T12:59:07.905 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-06-18 12:58:44.724963817 +0000 2024-06-18T12:59:07.905 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-06-18 12:53:22.286702996 +0000 2024-06-18T12:59:07.905 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-06-18 12:53:22.286702996 +0000 2024-06-18T12:59:07.905 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-06-18 12:53:22.286702996 +0000 2024-06-18T12:59:07.905 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-18T12:59:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-06-18T12:59:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-06-18T12:59:07.968 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000249424 s, 2.1 MB/s 2024-06-18T12:59:07.969 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-18T12:59:08.025 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_2 2024-06-18T12:59:08.080 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-18T12:59:08.081 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-18T12:59:08.081 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-06-18T12:59:08.081 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-18T12:59:08.081 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-06-18T12:59:08.081 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-06-18 12:58:44.724963817 +0000 2024-06-18T12:59:08.081 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-06-18 12:53:22.636695569 +0000 2024-06-18T12:59:08.081 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-06-18 12:53:22.636695569 +0000 2024-06-18T12:59:08.081 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-06-18 12:53:22.636695569 +0000 2024-06-18T12:59:08.081 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-18T12:59:08.146 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-06-18T12:59:08.146 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-06-18T12:59:08.146 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000187558 s, 2.7 MB/s 2024-06-18T12:59:08.148 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-18T12:59:08.204 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_3 2024-06-18T12:59:08.259 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-18T12:59:08.259 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-18T12:59:08.259 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-06-18T12:59:08.260 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-18T12:59:08.260 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-06-18T12:59:08.260 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-06-18 12:58:44.725963796 +0000 2024-06-18T12:59:08.260 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-06-18 12:53:22.980688269 +0000 2024-06-18T12:59:08.260 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-06-18 12:53:22.980688269 +0000 2024-06-18T12:59:08.260 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-06-18 12:53:22.980688269 +0000 2024-06-18T12:59:08.260 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-18T12:59:08.322 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-06-18T12:59:08.323 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-06-18T12:59:08.323 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000196404 s, 2.6 MB/s 2024-06-18T12:59:08.324 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-18T12:59:08.381 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_4 2024-06-18T12:59:08.436 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-18T12:59:08.437 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-18T12:59:08.437 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 727 Links: 1 2024-06-18T12:59:08.437 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-18T12:59:08.437 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-06-18T12:59:08.437 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-06-18 12:58:44.725963796 +0000 2024-06-18T12:59:08.437 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-06-18 12:53:23.321681032 +0000 2024-06-18T12:59:08.437 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-06-18 12:53:23.321681032 +0000 2024-06-18T12:59:08.437 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-06-18 12:53:23.321681032 +0000 2024-06-18T12:59:08.437 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-18T12:59:08.500 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:08.500 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:08.500 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:08.500 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:08.500 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:08.500 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:08.500 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-18T12:59:08.500 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:08 smithi179 ceph-mon[33312]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:08.500 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-18T12:59:08.501 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-18T12:59:08.501 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:08.501 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:08 smithi179 ceph-mon[33312]: Deploying daemon mgr.x on smithi179 2024-06-18T12:59:08.502 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-06-18T12:59:08.502 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-06-18T12:59:08.502 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000177826 s, 2.9 MB/s 2024-06-18T12:59:08.504 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-18T12:59:08.561 INFO:tasks.cephadm:Deploying osd.0 on smithi167 with /dev/vg_nvme/lv_4... 2024-06-18T12:59:08.561 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-18T12:59:08.803 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T12:59:08.926 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:08.926 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:08.926 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:08.926 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:08.926 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:08.926 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:08.927 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-18T12:59:08.927 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[29496]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:08.927 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-18T12:59:08.927 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-18T12:59:08.927 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:08.927 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[29496]: Deploying daemon mgr.x on smithi179 2024-06-18T12:59:08.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:08.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:08.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-18T12:59:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[37701]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-18T12:59:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-18T12:59:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:08 smithi167 ceph-mon[37701]: Deploying daemon mgr.x on smithi179 2024-06-18T12:59:09.769 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:09 smithi179 ceph-mon[33312]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:09.769 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:09 smithi179 systemd[1]: Started Ceph mgr.x for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T12:59:09.930 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:09 smithi167 ceph-mon[29496]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:09.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:09 smithi167 ceph-mon[37701]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:10.136 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:09.768+0000 7f76a3931040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-18T12:59:10.137 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:09.928+0000 7f76a3931040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-18T12:59:10.137 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:10 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:10.002+0000 7f76a3931040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-18T12:59:10.536 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-18T12:59:10.565 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:10 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:10.284+0000 7f76a3931040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-18T12:59:10.566 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:10 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:10.346+0000 7f76a3931040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-18T12:59:10.625 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-18T12:59:10.638 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-06-18T12:59:10.638 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-06-18T12:59:10.639 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113203 s, 926 MB/s 2024-06-18T12:59:10.639 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-06-18T12:59:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:10 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:10 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:10 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:10 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:10 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:10.974 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:10 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:10.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:10 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:10.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:10 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:10.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:10 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:10.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:10 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:10.975 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:10 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:10.975 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:10 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:10.975 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:10 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:10.975 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:10 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:10.975 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:10 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:11.295 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_4 2024-06-18T12:59:11.636 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:11.326+0000 7f76a3931040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-18T12:59:11.636 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:11.442+0000 7f76a3931040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-18T12:59:11.636 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:11.499+0000 7f76a3931040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-18T12:59:11.997 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:11.717+0000 7f76a3931040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-18T12:59:11.998 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:11.779+0000 7f76a3931040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-18T12:59:11.998 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:11.846+0000 7f76a3931040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-18T12:59:11.998 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:11.996+0000 7f76a3931040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-18T12:59:11.998 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:11.998 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:11.998 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:11.998 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:11.998 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:11.998 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-mon[33312]: Reconfiguring mgr.y (unknown last config time)... 2024-06-18T12:59:11.999 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-18T12:59:11.999 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-18T12:59:11.999 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:11.999 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-mon[33312]: Reconfiguring daemon mgr.y on smithi167 2024-06-18T12:59:11.999 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:11 smithi179 ceph-mon[33312]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:12.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:12.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:12.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:12.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:12.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:12.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[29496]: Reconfiguring mgr.y (unknown last config time)... 2024-06-18T12:59:12.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-18T12:59:12.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-18T12:59:12.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:12.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[29496]: Reconfiguring daemon mgr.y on smithi167 2024-06-18T12:59:12.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[29496]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:12.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:12.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:12.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:12.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:12.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:12.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[37701]: Reconfiguring mgr.y (unknown last config time)... 2024-06-18T12:59:12.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-18T12:59:12.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-18T12:59:12.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:12.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[37701]: Reconfiguring daemon mgr.y on smithi167 2024-06-18T12:59:12.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:11 smithi167 ceph-mon[37701]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:12.344 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T12:59:12.747 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:12 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:12.421+0000 7f76a3931040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-18T12:59:12.747 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:12 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:12.747+0000 7f76a3931040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-18T12:59:13.136 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:12 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:12.815+0000 7f76a3931040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-18T12:59:13.136 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:12 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:12.967+0000 7f76a3931040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-18T12:59:13.523 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:13.523 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:13.523 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:13.523 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:13.523 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:13.523 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:13 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:13 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:13 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:13.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:13 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:13.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:13 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:13.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:13 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:13.536 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:13 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:13.536 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:13 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:13.536 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:13 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:13.536 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:13 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:13.536 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:13 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:13.537 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:13 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:13.537 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:13 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:13.283+0000 7f76a3931040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-18T12:59:13.537 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:13 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:13.346+0000 7f76a3931040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-18T12:59:13.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.735+0000 7f21a3fc5640 1 -- 172.21.15.167:0/640739981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c100280 msgr2=0x7f219c100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:13.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.735+0000 7f21a3fc5640 1 --2- 172.21.15.167:0/640739981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c100280 0x7f219c100680 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2198009860 tx=0x7f219802a340 comp rx=0 tx=0).stop 2024-06-18T12:59:13.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.735+0000 7f21a3fc5640 1 -- 172.21.15.167:0/640739981 shutdown_connections 2024-06-18T12:59:13.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.735+0000 7f21a3fc5640 1 --2- 172.21.15.167:0/640739981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f219c101e40 0x7f219c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:13.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.735+0000 7f21a3fc5640 1 --2- 172.21.15.167:0/640739981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f219c101480 0x7f219c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:13.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.735+0000 7f21a3fc5640 1 --2- 172.21.15.167:0/640739981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c100280 0x7f219c100680 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:13.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.735+0000 7f21a3fc5640 1 -- 172.21.15.167:0/640739981 >> 172.21.15.167:0/640739981 conn(0x7f219c0fba30 msgr2=0x7f219c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:59:13.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.735+0000 7f21a3fc5640 1 -- 172.21.15.167:0/640739981 shutdown_connections 2024-06-18T12:59:13.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.735+0000 7f21a3fc5640 1 -- 172.21.15.167:0/640739981 wait complete. 2024-06-18T12:59:13.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.736+0000 7f21a3fc5640 1 Processor -- start 2024-06-18T12:59:13.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.736+0000 7f21a3fc5640 1 -- start start 2024-06-18T12:59:13.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.736+0000 7f21a3fc5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c100280 0x7f219c196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:13.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.736+0000 7f21a3fc5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f219c101480 0x7f219c1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:13.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.736+0000 7f21a3fc5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f219c101e40 0x7f219c19e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:13.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.736+0000 7f21a3fc5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f219c10b3b0 con 0x7f219c100280 2024-06-18T12:59:13.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.736+0000 7f21a3fc5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f219c10b230 con 0x7f219c101480 2024-06-18T12:59:13.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.736+0000 7f21a3fc5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f219c10b530 con 0x7f219c101e40 2024-06-18T12:59:13.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.736+0000 7f21a2562640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f219c101e40 0x7f219c19e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:13.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.736+0000 7f21a2562640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f219c101e40 0x7f219c19e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:41762/0 (socket says 172.21.15.167:41762) 2024-06-18T12:59:13.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.736+0000 7f21a1d61640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c100280 0x7f219c196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:13.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.736+0000 7f21a2562640 1 -- 172.21.15.167:0/2739188039 learned_addr learned my addr 172.21.15.167:0/2739188039 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:59:13.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.736+0000 7f21a1560640 1 --2- 172.21.15.167:0/2739188039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f219c101480 0x7f219c1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:13.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.737+0000 7f21a2562640 1 -- 172.21.15.167:0/2739188039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f219c101480 msgr2=0x7f219c1971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:13.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.737+0000 7f21a2562640 1 --2- 172.21.15.167:0/2739188039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f219c101480 0x7f219c1971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:13.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.737+0000 7f21a2562640 1 -- 172.21.15.167:0/2739188039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c100280 msgr2=0x7f219c196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:13.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.737+0000 7f21a2562640 1 --2- 172.21.15.167:0/2739188039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c100280 0x7f219c196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:13.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.737+0000 7f21a2562640 1 -- 172.21.15.167:0/2739188039 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c19e890 con 0x7f219c101e40 2024-06-18T12:59:13.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.737+0000 7f21a2562640 1 --2- 172.21.15.167:0/2739188039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f219c101e40 0x7f219c19e220 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f219400efc0 tx=0x7f219400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:59:13.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.737+0000 7f2192ffd640 1 -- 172.21.15.167:0/2739188039 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194003970 con 0x7f219c101e40 2024-06-18T12:59:13.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.737+0000 7f2192ffd640 1 -- 172.21.15.167:0/2739188039 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f2194004020 con 0x7f219c101e40 2024-06-18T12:59:13.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.737+0000 7f2192ffd640 1 -- 172.21.15.167:0/2739188039 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219400b870 con 0x7f219c101e40 2024-06-18T12:59:13.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.737+0000 7f21a3fc5640 1 -- 172.21.15.167:0/2739188039 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219c19eb80 con 0x7f219c101e40 2024-06-18T12:59:13.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.737+0000 7f21a3fc5640 1 -- 172.21.15.167:0/2739188039 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f219c19f040 con 0x7f219c101e40 2024-06-18T12:59:13.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.738+0000 7f2192ffd640 1 -- 172.21.15.167:0/2739188039 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f219401d020 con 0x7f219c101e40 2024-06-18T12:59:13.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.738+0000 7f21a3fc5640 1 -- 172.21.15.167:0/2739188039 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f219c100680 con 0x7f219c101e40 2024-06-18T12:59:13.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.739+0000 7f2192ffd640 1 --2- 172.21.15.167:0/2739188039 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f2188039290 0x7f218803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:13.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.739+0000 7f2192ffd640 1 -- 172.21.15.167:0/2739188039 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f219404c2e0 con 0x7f219c101e40 2024-06-18T12:59:13.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.739+0000 7f21a1d61640 1 --2- 172.21.15.167:0/2739188039 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f2188039290 0x7f218803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:13.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.739+0000 7f21a1d61640 1 --2- 172.21.15.167:0/2739188039 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f2188039290 0x7f218803b750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f219802a930 tx=0x7f2198030340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:59:13.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.744+0000 7f2192ffd640 1 -- 172.21.15.167:0/2739188039 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f219401ddf0 con 0x7f219c101e40 2024-06-18T12:59:13.838 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:13 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:13.536+0000 7f76a3931040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-18T12:59:13.839 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:13 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:13.687+0000 7f76a3931040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-18T12:59:13.839 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:13 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:13.753+0000 7f76a3931040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-18T12:59:13.839 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:13 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:13.838+0000 7f76a3931040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-18T12:59:13.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:13.936+0000 7f21a3fc5640 1 -- 172.21.15.167:0/2739188039 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f219c061de0 con 0x7f2188039290 2024-06-18T12:59:14.136 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:13 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:13.898+0000 7f76a3931040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-18T12:59:14.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:14 smithi167 ceph-mon[29496]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:14.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:14 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:14 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T12:59:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:14 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T12:59:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:14 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:14 smithi167 ceph-mon[37701]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:14 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:14 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T12:59:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:14 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T12:59:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:14 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:14.886 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:14 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:14.444+0000 7f76a3931040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-18T12:59:14.886 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 12:59:14 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T12:59:14.608+0000 7f76a3931040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-18T12:59:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:14 smithi179 ceph-mon[33312]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:14 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:14 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T12:59:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:14 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T12:59:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:14 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:15.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:15.418+0000 7f2192ffd640 1 -- 172.21.15.167:0/2739188039 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7f2194015c90 con 0x7f219c101e40 2024-06-18T12:59:15.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:15 smithi167 ceph-mon[37701]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:59:15.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:15 smithi167 ceph-mon[37701]: Standby manager daemon x started 2024-06-18T12:59:15.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:15 smithi167 ceph-mon[37701]: from='mgr.? 172.21.15.179:0/3672944808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-18T12:59:15.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:15 smithi167 ceph-mon[37701]: from='mgr.? 172.21.15.179:0/3672944808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-18T12:59:15.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:15 smithi167 ceph-mon[37701]: from='mgr.? 172.21.15.179:0/3672944808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-18T12:59:15.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:15 smithi167 ceph-mon[37701]: from='mgr.? 172.21.15.179:0/3672944808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-18T12:59:15.526 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:15 smithi167 ceph-mon[29496]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:59:15.526 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:15 smithi167 ceph-mon[29496]: Standby manager daemon x started 2024-06-18T12:59:15.526 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:15 smithi167 ceph-mon[29496]: from='mgr.? 172.21.15.179:0/3672944808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-18T12:59:15.526 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:15 smithi167 ceph-mon[29496]: from='mgr.? 172.21.15.179:0/3672944808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-18T12:59:15.527 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:15 smithi167 ceph-mon[29496]: from='mgr.? 172.21.15.179:0/3672944808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-18T12:59:15.527 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:15 smithi167 ceph-mon[29496]: from='mgr.? 172.21.15.179:0/3672944808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-18T12:59:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:15 smithi179 ceph-mon[33312]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:59:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:15 smithi179 ceph-mon[33312]: Standby manager daemon x started 2024-06-18T12:59:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:15 smithi179 ceph-mon[33312]: from='mgr.? 172.21.15.179:0/3672944808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-18T12:59:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:15 smithi179 ceph-mon[33312]: from='mgr.? 172.21.15.179:0/3672944808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-18T12:59:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:15 smithi179 ceph-mon[33312]: from='mgr.? 172.21.15.179:0/3672944808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-18T12:59:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:15 smithi179 ceph-mon[33312]: from='mgr.? 172.21.15.179:0/3672944808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-18T12:59:16.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:16 smithi167 ceph-mon[37701]: mgrmap e14: y(active, since 61s), standbys: x 2024-06-18T12:59:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:16 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-18T12:59:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:16 smithi167 ceph-mon[37701]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1283473259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df01b396-984c-4e66-83a8-e439554c688a"}]: dispatch 2024-06-18T12:59:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1283473259' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df01b396-984c-4e66-83a8-e439554c688a"}]': finished 2024-06-18T12:59:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:16 smithi167 ceph-mon[37701]: osdmap e5: 1 total, 0 up, 1 in 2024-06-18T12:59:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:16 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:16 smithi167 ceph-mon[29496]: mgrmap e14: y(active, since 61s), standbys: x 2024-06-18T12:59:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-18T12:59:16.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:16 smithi167 ceph-mon[29496]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:16.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1283473259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df01b396-984c-4e66-83a8-e439554c688a"}]: dispatch 2024-06-18T12:59:16.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1283473259' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df01b396-984c-4e66-83a8-e439554c688a"}]': finished 2024-06-18T12:59:16.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:16 smithi167 ceph-mon[29496]: osdmap e5: 1 total, 0 up, 1 in 2024-06-18T12:59:16.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:16.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:16 smithi179 ceph-mon[33312]: mgrmap e14: y(active, since 61s), standbys: x 2024-06-18T12:59:16.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:16 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-18T12:59:16.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:16 smithi179 ceph-mon[33312]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1283473259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df01b396-984c-4e66-83a8-e439554c688a"}]: dispatch 2024-06-18T12:59:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1283473259' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df01b396-984c-4e66-83a8-e439554c688a"}]': finished 2024-06-18T12:59:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:16 smithi179 ceph-mon[33312]: osdmap e5: 1 total, 0 up, 1 in 2024-06-18T12:59:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:16 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:17.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3549105836' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T12:59:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3549105836' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T12:59:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3549105836' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T12:59:18.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:18 smithi167 ceph-mon[29496]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:18 smithi167 ceph-mon[37701]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:18.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:18 smithi179 ceph-mon[33312]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:19.696 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:19 smithi167 ceph-mon[29496]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:19.696 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:19 smithi167 ceph-mon[37701]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:19.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:19 smithi179 ceph-mon[33312]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:21.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:21 smithi167 ceph-mon[29496]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:21 smithi167 ceph-mon[37701]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:21.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:21 smithi179 ceph-mon[33312]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:22.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:22 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-18T12:59:22.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:22 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:22.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:22 smithi179 ceph-mon[33312]: Deploying daemon osd.0 on smithi167 2024-06-18T12:59:22.893 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:22 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-18T12:59:22.893 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:22 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:22.893 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:22 smithi167 ceph-mon[29496]: Deploying daemon osd.0 on smithi167 2024-06-18T12:59:22.893 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:22 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-18T12:59:22.893 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:22 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:22.894 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:22 smithi167 ceph-mon[37701]: Deploying daemon osd.0 on smithi167 2024-06-18T12:59:23.868 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:23 smithi167 ceph-mon[29496]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:23.869 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:23 smithi167 ceph-mon[37701]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:23.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:23 smithi179 ceph-mon[33312]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:25.715 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:25 smithi167 ceph-mon[29496]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:25.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:25 smithi167 ceph-mon[37701]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:25.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:25 smithi179 ceph-mon[33312]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:27.614 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:27 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:27.614 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:27 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:27.615 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:27 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:27.615 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:27 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:27.615 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:27 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:27.615 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:27 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:27.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:27 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:27.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:27 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:27.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:27 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:28.528 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:28 smithi167 ceph-mon[29496]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:28.528 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:28 smithi167 ceph-mon[37701]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:28.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:28 smithi179 ceph-mon[33312]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:29.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:29 smithi167 ceph-mon[29496]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:29.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:29 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:29 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:29 smithi167 ceph-mon[37701]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:29 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:29 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:29.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:29 smithi179 ceph-mon[33312]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:29.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:29 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:29.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:29 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:30.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:30.398+0000 7f2192ffd640 1 -- 172.21.15.167:0/2739188039 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f219c061de0 con 0x7f2188039290 2024-06-18T12:59:30.398 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 0 on host 'smithi167' 2024-06-18T12:59:30.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:30.400+0000 7f21a3fc5640 1 -- 172.21.15.167:0/2739188039 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f2188039290 msgr2=0x7f218803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:30.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:30.400+0000 7f21a3fc5640 1 --2- 172.21.15.167:0/2739188039 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f2188039290 0x7f218803b750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f219802a930 tx=0x7f2198030340 comp rx=0 tx=0).stop 2024-06-18T12:59:30.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:30.400+0000 7f21a3fc5640 1 -- 172.21.15.167:0/2739188039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f219c101e40 msgr2=0x7f219c19e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:30.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:30.400+0000 7f21a3fc5640 1 --2- 172.21.15.167:0/2739188039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f219c101e40 0x7f219c19e220 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f219400efc0 tx=0x7f219400c800 comp rx=0 tx=0).stop 2024-06-18T12:59:30.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:30.400+0000 7f21a3fc5640 1 -- 172.21.15.167:0/2739188039 shutdown_connections 2024-06-18T12:59:30.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:30.400+0000 7f21a3fc5640 1 --2- 172.21.15.167:0/2739188039 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f2188039290 0x7f218803b750 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:30.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:30.400+0000 7f21a3fc5640 1 --2- 172.21.15.167:0/2739188039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f219c101e40 0x7f219c19e220 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:30.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:30.400+0000 7f21a3fc5640 1 --2- 172.21.15.167:0/2739188039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f219c101480 0x7f219c1971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:30.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:30.400+0000 7f21a3fc5640 1 --2- 172.21.15.167:0/2739188039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c100280 0x7f219c196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:30.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:30.400+0000 7f21a3fc5640 1 -- 172.21.15.167:0/2739188039 >> 172.21.15.167:0/2739188039 conn(0x7f219c0fba30 msgr2=0x7f219c069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:59:30.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:30.401+0000 7f21a3fc5640 1 -- 172.21.15.167:0/2739188039 shutdown_connections 2024-06-18T12:59:30.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:30.401+0000 7f21a3fc5640 1 -- 172.21.15.167:0/2739188039 wait complete. 2024-06-18T12:59:30.690 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:30 smithi167 ceph-mon[37701]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-18T12:59:30.690 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:30 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:30.690 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:30 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:30.690 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:30 smithi167 ceph-mon[29496]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-18T12:59:30.691 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:30 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:30.691 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:30 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:30.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:30 smithi179 ceph-mon[33312]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-18T12:59:30.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:30 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:30.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:30 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:30.959 DEBUG:teuthology.orchestra.run.smithi167:osd.0> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.0.service 2024-06-18T12:59:30.962 INFO:tasks.cephadm:Deploying osd.1 on smithi167 with /dev/vg_nvme/lv_3... 2024-06-18T12:59:30.962 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-18T12:59:31.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:31 smithi179 ceph-mon[33312]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-18T12:59:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:31 smithi179 ceph-mon[33312]: osdmap e6: 1 total, 0 up, 1 in 2024-06-18T12:59:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:31 smithi179 ceph-mon[33312]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T12:59:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:31 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:31 smithi179 ceph-mon[33312]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:31 smithi167 ceph-mon[29496]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-18T12:59:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:31 smithi167 ceph-mon[29496]: osdmap e6: 1 total, 0 up, 1 in 2024-06-18T12:59:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:31 smithi167 ceph-mon[29496]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T12:59:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:31 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:31 smithi167 ceph-mon[29496]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:31.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:31 smithi167 ceph-mon[37701]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-18T12:59:31.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:31 smithi167 ceph-mon[37701]: osdmap e6: 1 total, 0 up, 1 in 2024-06-18T12:59:31.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:31 smithi167 ceph-mon[37701]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T12:59:31.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:31 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:31.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:31 smithi167 ceph-mon[37701]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:32.149 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T12:59:32.355 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 12:59:32 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0[41556]: 2024-06-18T12:59:32.020+0000 7f163112e640 -1 osd.0 0 waiting for initial osdmap 2024-06-18T12:59:32.355 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 12:59:32 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0[41556]: 2024-06-18T12:59:32.023+0000 7f1628f4e640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-18T12:59:32.670 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:32 smithi167 ceph-mon[29496]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-18T12:59:32.670 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:32 smithi167 ceph-mon[29496]: osdmap e7: 1 total, 0 up, 1 in 2024-06-18T12:59:32.670 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:32 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:32.670 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:32 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:32.670 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:32 smithi167 ceph-mon[29496]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' 2024-06-18T12:59:32.671 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:32 smithi167 ceph-mon[37701]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-18T12:59:32.671 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:32 smithi167 ceph-mon[37701]: osdmap e7: 1 total, 0 up, 1 in 2024-06-18T12:59:32.671 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:32 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:32.671 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:32 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:32.671 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:32 smithi167 ceph-mon[37701]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' 2024-06-18T12:59:32.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:32 smithi179 ceph-mon[33312]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-18T12:59:32.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:32 smithi179 ceph-mon[33312]: osdmap e7: 1 total, 0 up, 1 in 2024-06-18T12:59:32.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:32 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:32.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:32 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:32.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:32 smithi179 ceph-mon[33312]: from='osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419]' entity='osd.0' 2024-06-18T12:59:33.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:33 smithi167 ceph-mon[29496]: purged_snaps scrub starts 2024-06-18T12:59:33.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:33 smithi167 ceph-mon[29496]: purged_snaps scrub ok 2024-06-18T12:59:33.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:33 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:33.822 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:33 smithi167 ceph-mon[29496]: osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419] boot 2024-06-18T12:59:33.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:33 smithi167 ceph-mon[29496]: osdmap e8: 1 total, 1 up, 1 in 2024-06-18T12:59:33.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:33 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:33.823 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:33 smithi167 ceph-mon[29496]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:33.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:33 smithi167 ceph-mon[37701]: purged_snaps scrub starts 2024-06-18T12:59:33.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:33 smithi167 ceph-mon[37701]: purged_snaps scrub ok 2024-06-18T12:59:33.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:33 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:33.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:33 smithi167 ceph-mon[37701]: osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419] boot 2024-06-18T12:59:33.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:33 smithi167 ceph-mon[37701]: osdmap e8: 1 total, 1 up, 1 in 2024-06-18T12:59:33.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:33 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:33.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:33 smithi167 ceph-mon[37701]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:33.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:33 smithi179 ceph-mon[33312]: purged_snaps scrub starts 2024-06-18T12:59:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:33 smithi179 ceph-mon[33312]: purged_snaps scrub ok 2024-06-18T12:59:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:33 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:33 smithi179 ceph-mon[33312]: osd.0 [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419] boot 2024-06-18T12:59:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:33 smithi179 ceph-mon[33312]: osdmap e8: 1 total, 1 up, 1 in 2024-06-18T12:59:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:33 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T12:59:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:33 smithi179 ceph-mon[33312]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-18T12:59:34.927 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-18T12:59:35.012 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-18T12:59:35.026 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-06-18T12:59:35.026 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-06-18T12:59:35.026 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123768 s, 847 MB/s 2024-06-18T12:59:35.026 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-06-18T12:59:35.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:34 smithi179 ceph-mon[33312]: Detected new or changed devices on smithi167 2024-06-18T12:59:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:34 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:34 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:34 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T12:59:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:34 smithi179 ceph-mon[33312]: Adjusting osd_memory_target on smithi167 to 16064M 2024-06-18T12:59:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:34 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:34 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:34 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:34 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:34 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:34 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:34 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:35.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:34 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:34 smithi179 ceph-mon[33312]: osdmap e9: 1 total, 1 up, 1 in 2024-06-18T12:59:35.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[37701]: Detected new or changed devices on smithi167 2024-06-18T12:59:35.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T12:59:35.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[37701]: Adjusting osd_memory_target on smithi167 to 16064M 2024-06-18T12:59:35.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:35.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:35.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:35.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:35.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:35.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[37701]: osdmap e9: 1 total, 1 up, 1 in 2024-06-18T12:59:35.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[29496]: Detected new or changed devices on smithi167 2024-06-18T12:59:35.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T12:59:35.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[29496]: Adjusting osd_memory_target on smithi167 to 16064M 2024-06-18T12:59:35.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:35.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:35.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:35.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:35.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:35.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:35.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:34 smithi167 ceph-mon[29496]: osdmap e9: 1 total, 1 up, 1 in 2024-06-18T12:59:35.552 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_3 2024-06-18T12:59:35.787 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T12:59:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:35 smithi179 ceph-mon[33312]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:36.153 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:35 smithi167 ceph-mon[29496]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:36.153 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:35 smithi167 ceph-mon[37701]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:37.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.136+0000 7fe1061fd640 1 -- 172.21.15.167:0/1081729169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1001014a0 msgr2=0x7fe100101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:37.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.136+0000 7fe1061fd640 1 --2- 172.21.15.167:0/1081729169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1001014a0 0x7fe100101920 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009860 tx=0x7fe0e802a340 comp rx=0 tx=0).stop 2024-06-18T12:59:37.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.136+0000 7fe1061fd640 1 -- 172.21.15.167:0/1081729169 shutdown_connections 2024-06-18T12:59:37.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.136+0000 7fe1061fd640 1 --2- 172.21.15.167:0/1081729169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe100101e60 0x7fe1001086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:37.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.136+0000 7fe1061fd640 1 --2- 172.21.15.167:0/1081729169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1001014a0 0x7fe100101920 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:37.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.136+0000 7fe1061fd640 1 --2- 172.21.15.167:0/1081729169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1001002a0 0x7fe1001006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:37.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.136+0000 7fe1061fd640 1 -- 172.21.15.167:0/1081729169 >> 172.21.15.167:0/1081729169 conn(0x7fe1000fba30 msgr2=0x7fe1000fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:59:37.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.136+0000 7fe1061fd640 1 -- 172.21.15.167:0/1081729169 shutdown_connections 2024-06-18T12:59:37.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.136+0000 7fe1061fd640 1 -- 172.21.15.167:0/1081729169 wait complete. 2024-06-18T12:59:37.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.136+0000 7fe1061fd640 1 Processor -- start 2024-06-18T12:59:37.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.136+0000 7fe1061fd640 1 -- start start 2024-06-18T12:59:37.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe1061fd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1001002a0 0x7fe100104230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:37.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe1061fd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1001014a0 0x7fe100102880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:37.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe1061fd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe100101e60 0x7fe100102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:37.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe1061fd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe10010b410 con 0x7fe1001002a0 2024-06-18T12:59:37.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe1061fd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe10010b290 con 0x7fe1001014a0 2024-06-18T12:59:37.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe1061fd640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe10010b590 con 0x7fe100101e60 2024-06-18T12:59:37.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0ff7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1001002a0 0x7fe100104230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:37.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0fffff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe100101e60 0x7fe100102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:37.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0ff7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1001002a0 0x7fe100104230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34460/0 (socket says 172.21.15.167:34460) 2024-06-18T12:59:37.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0ff7fe640 1 -- 172.21.15.167:0/4157379565 learned_addr learned my addr 172.21.15.167:0/4157379565 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T12:59:37.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0f6ffd640 1 --2- 172.21.15.167:0/4157379565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1001014a0 0x7fe100102880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:37.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0ff7fe640 1 -- 172.21.15.167:0/4157379565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe100101e60 msgr2=0x7fe100102dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:37.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0ff7fe640 1 --2- 172.21.15.167:0/4157379565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe100101e60 0x7fe100102dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:37.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0ff7fe640 1 -- 172.21.15.167:0/4157379565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1001014a0 msgr2=0x7fe100102880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:37.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0ff7fe640 1 --2- 172.21.15.167:0/4157379565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1001014a0 0x7fe100102880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:37.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0ff7fe640 1 -- 172.21.15.167:0/4157379565 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe100103680 con 0x7fe1001002a0 2024-06-18T12:59:37.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0fffff640 1 --2- 172.21.15.167:0/4157379565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe100101e60 0x7fe100102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T12:59:37.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0ff7fe640 1 --2- 172.21.15.167:0/4157379565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1001002a0 0x7fe100104230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000e9a0 tx=0x7fe0f000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:59:37.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0fd7fa640 1 -- 172.21.15.167:0/4157379565 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f000cd80 con 0x7fe1001002a0 2024-06-18T12:59:37.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0fd7fa640 1 -- 172.21.15.167:0/4157379565 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fe0f00108b0 con 0x7fe1001002a0 2024-06-18T12:59:37.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.137+0000 7fe0fd7fa640 1 -- 172.21.15.167:0/4157379565 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0018830 con 0x7fe1001002a0 2024-06-18T12:59:37.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.138+0000 7fe1061fd640 1 -- 172.21.15.167:0/4157379565 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1001a0e40 con 0x7fe1001002a0 2024-06-18T12:59:37.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.138+0000 7fe1061fd640 1 -- 172.21.15.167:0/4157379565 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1001a11e0 con 0x7fe1001002a0 2024-06-18T12:59:37.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.139+0000 7fe1061fd640 1 -- 172.21.15.167:0/4157379565 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1001006a0 con 0x7fe1001002a0 2024-06-18T12:59:37.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.139+0000 7fe0fd7fa640 1 -- 172.21.15.167:0/4157379565 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7fe0f0010430 con 0x7fe1001002a0 2024-06-18T12:59:37.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.139+0000 7fe0fd7fa640 1 --2- 172.21.15.167:0/4157379565 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe0ec06dec0 0x7fe0ec070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T12:59:37.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.139+0000 7fe0fd7fa640 1 -- 172.21.15.167:0/4157379565 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fe0f008d1e0 con 0x7fe1001002a0 2024-06-18T12:59:37.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.139+0000 7fe0f6ffd640 1 --2- 172.21.15.167:0/4157379565 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe0ec06dec0 0x7fe0ec070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T12:59:37.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.140+0000 7fe0f6ffd640 1 --2- 172.21.15.167:0/4157379565 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe0ec06dec0 0x7fe0ec070380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8000c00 tx=0x7fe0e8030340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T12:59:37.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.144+0000 7fe0fd7fa640 1 -- 172.21.15.167:0/4157379565 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fe0f00538c0 con 0x7fe1001002a0 2024-06-18T12:59:37.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:37.328+0000 7fe1061fd640 1 -- 172.21.15.167:0/4157379565 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe10002d690 con 0x7fe0ec06dec0 2024-06-18T12:59:37.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:37 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T12:59:37.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:37 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T12:59:37.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:37 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:37.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:37 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T12:59:37.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:37 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T12:59:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:37 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:37 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T12:59:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:37 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T12:59:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:37 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:38.499 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:38 smithi167 ceph-mon[29496]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:59:38.499 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:38 smithi167 ceph-mon[29496]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:38.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:38 smithi167 ceph-mon[37701]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:59:38.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:38 smithi167 ceph-mon[37701]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:38.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:38 smithi179 ceph-mon[33312]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T12:59:38.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:38 smithi179 ceph-mon[33312]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:39.837 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:39 smithi167 ceph-mon[37701]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:39.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1840275733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8159aff-9414-4ebe-897f-ffbf27ed9e7f"}]: dispatch 2024-06-18T12:59:39.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1840275733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8159aff-9414-4ebe-897f-ffbf27ed9e7f"}]': finished 2024-06-18T12:59:39.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:39 smithi167 ceph-mon[37701]: osdmap e10: 2 total, 1 up, 2 in 2024-06-18T12:59:39.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:39 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:39.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:39 smithi167 ceph-mon[29496]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:39.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1840275733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8159aff-9414-4ebe-897f-ffbf27ed9e7f"}]: dispatch 2024-06-18T12:59:39.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1840275733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8159aff-9414-4ebe-897f-ffbf27ed9e7f"}]': finished 2024-06-18T12:59:39.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:39 smithi167 ceph-mon[29496]: osdmap e10: 2 total, 1 up, 2 in 2024-06-18T12:59:39.838 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:39 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:39.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:39 smithi179 ceph-mon[33312]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:39.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1840275733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8159aff-9414-4ebe-897f-ffbf27ed9e7f"}]: dispatch 2024-06-18T12:59:39.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1840275733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8159aff-9414-4ebe-897f-ffbf27ed9e7f"}]': finished 2024-06-18T12:59:39.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:39 smithi179 ceph-mon[33312]: osdmap e10: 2 total, 1 up, 2 in 2024-06-18T12:59:39.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:39 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3821801032' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T12:59:40.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3821801032' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T12:59:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3821801032' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T12:59:41.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:41 smithi179 ceph-mon[33312]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:41.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:41 smithi167 ceph-mon[29496]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:41.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:41 smithi167 ceph-mon[37701]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:43.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:43 smithi179 ceph-mon[33312]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:43.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:43 smithi167 ceph-mon[37701]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:43.931 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:43 smithi167 ceph-mon[29496]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:45.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:45 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-18T12:59:45.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:45 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:45.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:45 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-18T12:59:45.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:45 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:45.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:45 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-18T12:59:45.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:45 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:46.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:46 smithi167 ceph-mon[29496]: Deploying daemon osd.1 on smithi167 2024-06-18T12:59:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:46 smithi167 ceph-mon[29496]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:46 smithi167 ceph-mon[37701]: Deploying daemon osd.1 on smithi167 2024-06-18T12:59:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:46 smithi167 ceph-mon[37701]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:46.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:46 smithi179 ceph-mon[33312]: Deploying daemon osd.1 on smithi167 2024-06-18T12:59:46.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:46 smithi179 ceph-mon[33312]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:47.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:47 smithi167 ceph-mon[37701]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:47.834 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:47 smithi167 ceph-mon[29496]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:47.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:47 smithi179 ceph-mon[33312]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:49.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:49 smithi179 ceph-mon[33312]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:49.970 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:49 smithi167 ceph-mon[29496]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:49.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:49 smithi167 ceph-mon[37701]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:50.860 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:50 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:50.860 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:50 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:50.860 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:50 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:50.860 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:50 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:50.860 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:50 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:50.860 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:50 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:50.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:50 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:50.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:50 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:50.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:50 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:51.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:51 smithi179 ceph-mon[33312]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:51.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:51 smithi167 ceph-mon[29496]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:51.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:51 smithi167 ceph-mon[37701]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:53.690 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:53.690 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:53.690 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:53.691 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:53.691 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:53.691 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[29496]: from='osd.1 [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-18T12:59:53.691 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[29496]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-18T12:59:53.691 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[29496]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:53.691 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:53.691 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:53.691 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:53.691 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:53.691 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:53.691 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[37701]: from='osd.1 [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-18T12:59:53.692 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[37701]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-18T12:59:53.692 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:53 smithi167 ceph-mon[37701]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:53.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:53.727+0000 7fe0fd7fa640 1 -- 172.21.15.167:0/4157379565 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe10002d690 con 0x7fe0ec06dec0 2024-06-18T12:59:53.729 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 1 on host 'smithi167' 2024-06-18T12:59:53.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:53.729+0000 7fe1061fd640 1 -- 172.21.15.167:0/4157379565 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe0ec06dec0 msgr2=0x7fe0ec070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:53.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:53.729+0000 7fe1061fd640 1 --2- 172.21.15.167:0/4157379565 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe0ec06dec0 0x7fe0ec070380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8000c00 tx=0x7fe0e8030340 comp rx=0 tx=0).stop 2024-06-18T12:59:53.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:53.729+0000 7fe1061fd640 1 -- 172.21.15.167:0/4157379565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1001002a0 msgr2=0x7fe100104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T12:59:53.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:53.729+0000 7fe1061fd640 1 --2- 172.21.15.167:0/4157379565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1001002a0 0x7fe100104230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000e9a0 tx=0x7fe0f000ee60 comp rx=0 tx=0).stop 2024-06-18T12:59:53.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:53.730+0000 7fe1061fd640 1 -- 172.21.15.167:0/4157379565 shutdown_connections 2024-06-18T12:59:53.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:53.730+0000 7fe1061fd640 1 --2- 172.21.15.167:0/4157379565 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fe0ec06dec0 0x7fe0ec070380 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:53.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:53.730+0000 7fe1061fd640 1 --2- 172.21.15.167:0/4157379565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe100101e60 0x7fe100102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:53.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:53.730+0000 7fe1061fd640 1 --2- 172.21.15.167:0/4157379565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1001014a0 0x7fe100102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:53.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:53.730+0000 7fe1061fd640 1 --2- 172.21.15.167:0/4157379565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1001002a0 0x7fe100104230 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T12:59:53.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:53.730+0000 7fe1061fd640 1 -- 172.21.15.167:0/4157379565 >> 172.21.15.167:0/4157379565 conn(0x7fe1000fba30 msgr2=0x7fe10006a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T12:59:53.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:53.730+0000 7fe1061fd640 1 -- 172.21.15.167:0/4157379565 shutdown_connections 2024-06-18T12:59:53.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T12:59:53.730+0000 7fe1061fd640 1 -- 172.21.15.167:0/4157379565 wait complete. 2024-06-18T12:59:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:53 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:53 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:53 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:53 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:53 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:54.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:53 smithi179 ceph-mon[33312]: from='osd.1 [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-18T12:59:54.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:53 smithi179 ceph-mon[33312]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-18T12:59:54.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:53 smithi179 ceph-mon[33312]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:54.315 DEBUG:teuthology.orchestra.run.smithi167:osd.1> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.1.service 2024-06-18T12:59:54.317 INFO:tasks.cephadm:Deploying osd.2 on smithi167 with /dev/vg_nvme/lv_2... 2024-06-18T12:59:54.317 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-18T12:59:54.961 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T12:59:54.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[37701]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-18T12:59:54.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[37701]: osdmap e11: 2 total, 1 up, 2 in 2024-06-18T12:59:54.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:54.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[37701]: from='osd.1 [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T12:59:54.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[37701]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T12:59:54.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[29496]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-18T12:59:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[29496]: osdmap e11: 2 total, 1 up, 2 in 2024-06-18T12:59:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[29496]: from='osd.1 [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T12:59:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[29496]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T12:59:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:54 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:55.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:54 smithi179 ceph-mon[33312]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-18T12:59:55.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:54 smithi179 ceph-mon[33312]: osdmap e11: 2 total, 1 up, 2 in 2024-06-18T12:59:55.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:54 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:54 smithi179 ceph-mon[33312]: from='osd.1 [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T12:59:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:54 smithi179 ceph-mon[33312]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T12:59:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:54 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T12:59:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:54 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:54 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:55.476 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 12:59:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-1[47148]: 2024-06-18T12:59:55.097+0000 7f08675d9640 -1 osd.1 0 waiting for initial osdmap 2024-06-18T12:59:55.476 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 12:59:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-1[47148]: 2024-06-18T12:59:55.100+0000 7f085f3f9640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-18T12:59:55.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:55 smithi167 ceph-mon[37701]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-18T12:59:55.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:55 smithi167 ceph-mon[37701]: osdmap e12: 2 total, 1 up, 2 in 2024-06-18T12:59:55.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:55.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:55.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:55 smithi167 ceph-mon[37701]: from='osd.1 ' entity='osd.1' 2024-06-18T12:59:55.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:55 smithi167 ceph-mon[37701]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:55.839 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:55 smithi167 ceph-mon[29496]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-18T12:59:55.839 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:55 smithi167 ceph-mon[29496]: osdmap e12: 2 total, 1 up, 2 in 2024-06-18T12:59:55.840 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:55.840 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:55.840 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:55 smithi167 ceph-mon[29496]: from='osd.1 ' entity='osd.1' 2024-06-18T12:59:55.840 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:55 smithi167 ceph-mon[29496]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:56.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:55 smithi179 ceph-mon[33312]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-18T12:59:56.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:55 smithi179 ceph-mon[33312]: osdmap e12: 2 total, 1 up, 2 in 2024-06-18T12:59:56.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:56.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:56.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:55 smithi179 ceph-mon[33312]: from='osd.1 ' entity='osd.1' 2024-06-18T12:59:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:55 smithi179 ceph-mon[33312]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-18T12:59:56.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:56 smithi167 ceph-mon[37701]: purged_snaps scrub starts 2024-06-18T12:59:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:56 smithi167 ceph-mon[37701]: purged_snaps scrub ok 2024-06-18T12:59:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:56 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:56 smithi167 ceph-mon[37701]: osd.1 [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972] boot 2024-06-18T12:59:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:56 smithi167 ceph-mon[37701]: osdmap e13: 2 total, 2 up, 2 in 2024-06-18T12:59:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:56 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:56.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:56 smithi167 ceph-mon[29496]: purged_snaps scrub starts 2024-06-18T12:59:56.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:56 smithi167 ceph-mon[29496]: purged_snaps scrub ok 2024-06-18T12:59:56.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:56 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:56.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:56 smithi167 ceph-mon[29496]: osd.1 [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972] boot 2024-06-18T12:59:56.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:56 smithi167 ceph-mon[29496]: osdmap e13: 2 total, 2 up, 2 in 2024-06-18T12:59:56.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:56 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:57.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:56 smithi179 ceph-mon[33312]: purged_snaps scrub starts 2024-06-18T12:59:57.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:56 smithi179 ceph-mon[33312]: purged_snaps scrub ok 2024-06-18T12:59:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:56 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:56 smithi179 ceph-mon[33312]: osd.1 [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972] boot 2024-06-18T12:59:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:56 smithi179 ceph-mon[33312]: osdmap e13: 2 total, 2 up, 2 in 2024-06-18T12:59:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:56 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T12:59:58.141 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-18T12:59:58.236 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-18T12:59:58.256 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-06-18T12:59:58.257 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-06-18T12:59:58.257 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.01833 s, 572 MB/s 2024-06-18T12:59:58.257 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-06-18T12:59:58.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:58 smithi179 ceph-mon[33312]: osdmap e14: 2 total, 2 up, 2 in 2024-06-18T12:59:58.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:58 smithi179 ceph-mon[33312]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T12:59:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:58 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:58 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:58 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T12:59:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:58 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T12:59:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:58 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:58 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:58 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:58 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:58.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[29496]: osdmap e14: 2 total, 2 up, 2 in 2024-06-18T12:59:58.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[29496]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T12:59:58.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:58.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:58.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T12:59:58.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T12:59:58.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:58.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:58.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:58.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:58.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[37701]: osdmap e14: 2 total, 2 up, 2 in 2024-06-18T12:59:58.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[37701]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T12:59:58.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:58.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:58.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T12:59:58.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T12:59:58.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:58.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T12:59:58.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T12:59:58.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:58 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T12:59:58.848 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_2 2024-06-18T12:59:59.084 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T12:59:59.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:59 smithi179 ceph-mon[33312]: Detected new or changed devices on smithi167 2024-06-18T12:59:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 12:59:59 smithi179 ceph-mon[33312]: Adjusting osd_memory_target on smithi167 to 8032M 2024-06-18T12:59:59.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:59 smithi167 ceph-mon[37701]: Detected new or changed devices on smithi167 2024-06-18T12:59:59.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 12:59:59 smithi167 ceph-mon[37701]: Adjusting osd_memory_target on smithi167 to 8032M 2024-06-18T12:59:59.432 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:59 smithi167 ceph-mon[29496]: Detected new or changed devices on smithi167 2024-06-18T12:59:59.432 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 12:59:59 smithi167 ceph-mon[29496]: Adjusting osd_memory_target on smithi167 to 8032M 2024-06-18T13:00:00.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:00 smithi179 ceph-mon[33312]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:00.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:00 smithi179 ceph-mon[33312]: overall HEALTH_OK 2024-06-18T13:00:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:00 smithi167 ceph-mon[37701]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:00 smithi167 ceph-mon[37701]: overall HEALTH_OK 2024-06-18T13:00:00.429 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:00 smithi167 ceph-mon[29496]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:00.429 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:00 smithi167 ceph-mon[29496]: overall HEALTH_OK 2024-06-18T13:00:00.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.616+0000 7ff4a92c3640 1 -- 172.21.15.167:0/4205928253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4a41013c0 msgr2=0x7ff4a41017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:00:00.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.616+0000 7ff4a92c3640 1 --2- 172.21.15.167:0/4205928253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4a41013c0 0x7ff4a41017c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff494009900 tx=0x7ff49402a410 comp rx=0 tx=0).stop 2024-06-18T13:00:00.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.616+0000 7ff4a92c3640 1 -- 172.21.15.167:0/4205928253 shutdown_connections 2024-06-18T13:00:00.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.616+0000 7ff4a92c3640 1 --2- 172.21.15.167:0/4205928253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a4102f80 0x7ff4a4109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:00.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.616+0000 7ff4a92c3640 1 --2- 172.21.15.167:0/4205928253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a41025c0 0x7ff4a4102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:00.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.616+0000 7ff4a92c3640 1 --2- 172.21.15.167:0/4205928253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4a41013c0 0x7ff4a41017c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:00.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.616+0000 7ff4a92c3640 1 -- 172.21.15.167:0/4205928253 >> 172.21.15.167:0/4205928253 conn(0x7ff4a40fcb70 msgr2=0x7ff4a40fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:00:00.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.616+0000 7ff4a92c3640 1 -- 172.21.15.167:0/4205928253 shutdown_connections 2024-06-18T13:00:00.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.616+0000 7ff4a92c3640 1 -- 172.21.15.167:0/4205928253 wait complete. 2024-06-18T13:00:00.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.616+0000 7ff4a92c3640 1 Processor -- start 2024-06-18T13:00:00.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a92c3640 1 -- start start 2024-06-18T13:00:00.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a92c3640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4a41013c0 0x7ff4a4196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:00:00.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a92c3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a41025c0 0x7ff4a4197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:00:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a92c3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a4102f80 0x7ff4a419e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:00:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a92c3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a410c4f0 con 0x7ff4a4102f80 2024-06-18T13:00:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a92c3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a410c370 con 0x7ff4a41025c0 2024-06-18T13:00:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a92c3640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a410c670 con 0x7ff4a41013c0 2024-06-18T13:00:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a2ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4a41013c0 0x7ff4a4196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:00:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a2ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4a41013c0 0x7ff4a4196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:42400/0 (socket says 172.21.15.167:42400) 2024-06-18T13:00:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a2ffd640 1 -- 172.21.15.167:0/3538136236 learned_addr learned my addr 172.21.15.167:0/3538136236 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:00:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a37fe640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a4102f80 0x7ff4a419e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:00:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a27fc640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a41025c0 0x7ff4a4197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:00:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a2ffd640 1 -- 172.21.15.167:0/3538136236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a41025c0 msgr2=0x7ff4a4197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:00:00.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a2ffd640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a41025c0 0x7ff4a4197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:00.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a2ffd640 1 -- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a4102f80 msgr2=0x7ff4a419e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:00:00.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a2ffd640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a4102f80 0x7ff4a419e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:00.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a2ffd640 1 -- 172.21.15.167:0/3538136236 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a419e850 con 0x7ff4a41013c0 2024-06-18T13:00:00.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a37fe640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a4102f80 0x7ff4a419e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:00:00.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.617+0000 7ff4a2ffd640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4a41013c0 0x7ff4a4196c30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff494005940 tx=0x7ff494005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:00:00.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.618+0000 7ff48bfff640 1 -- 172.21.15.167:0/3538136236 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49403f070 con 0x7ff4a41013c0 2024-06-18T13:00:00.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.618+0000 7ff4a92c3640 1 -- 172.21.15.167:0/3538136236 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a419eae0 con 0x7ff4a41013c0 2024-06-18T13:00:00.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.618+0000 7ff48bfff640 1 -- 172.21.15.167:0/3538136236 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7ff494003a70 con 0x7ff4a41013c0 2024-06-18T13:00:00.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.618+0000 7ff48bfff640 1 -- 172.21.15.167:0/3538136236 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494035030 con 0x7ff4a41013c0 2024-06-18T13:00:00.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.618+0000 7ff4a92c3640 1 -- 172.21.15.167:0/3538136236 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a419f010 con 0x7ff4a41013c0 2024-06-18T13:00:00.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.619+0000 7ff4a92c3640 1 -- 172.21.15.167:0/3538136236 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a41017c0 con 0x7ff4a41013c0 2024-06-18T13:00:00.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.619+0000 7ff48bfff640 1 -- 172.21.15.167:0/3538136236 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7ff494042070 con 0x7ff4a41013c0 2024-06-18T13:00:00.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.620+0000 7ff48bfff640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff48c06dec0 0x7ff48c070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:00:00.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.620+0000 7ff48bfff640 1 -- 172.21.15.167:0/3538136236 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(14..14 src has 1..14) ==== 2199+0+0 (secure 0 0 0) 0x7ff4940ad7d0 con 0x7ff4a41013c0 2024-06-18T13:00:00.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.620+0000 7ff4a27fc640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff48c06dec0 0x7ff48c070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:00:00.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.620+0000 7ff4a27fc640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff48c06dec0 0x7ff48c070380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff49800bbe0 tx=0x7ff49800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:00:00.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.624+0000 7ff48bfff640 1 -- 172.21.15.167:0/3538136236 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7ff49407fe40 con 0x7ff4a41013c0 2024-06-18T13:00:00.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:00.811+0000 7ff4a92c3640 1 -- 172.21.15.167:0/3538136236 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff4a402d590 con 0x7ff48c06dec0 2024-06-18T13:00:01.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:01 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:00:01.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:01 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:00:01.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:01 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:01.393 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:01 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:00:01.393 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:01 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:00:01.393 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:01 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:01.394 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:01 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:00:01.394 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:01 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:00:01.394 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:01 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:02.376 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:02 smithi167 ceph-mon[37701]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:00:02.376 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:02 smithi167 ceph-mon[37701]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:02.376 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:02 smithi167 ceph-mon[29496]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:00:02.376 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:02 smithi167 ceph-mon[29496]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:02.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:02 smithi179 ceph-mon[33312]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:00:02.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:02 smithi179 ceph-mon[33312]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:03.391 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:03 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4206571389' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81482f44-3d16-4c10-b2ac-88eeeff554fd"}]: dispatch 2024-06-18T13:00:03.391 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:03 smithi167 ceph-mon[29496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81482f44-3d16-4c10-b2ac-88eeeff554fd"}]: dispatch 2024-06-18T13:00:03.391 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:03 smithi167 ceph-mon[29496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81482f44-3d16-4c10-b2ac-88eeeff554fd"}]': finished 2024-06-18T13:00:03.391 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:03 smithi167 ceph-mon[29496]: osdmap e15: 3 total, 2 up, 3 in 2024-06-18T13:00:03.391 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:03 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:03.391 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:03 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4206571389' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81482f44-3d16-4c10-b2ac-88eeeff554fd"}]: dispatch 2024-06-18T13:00:03.391 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:03 smithi167 ceph-mon[37701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81482f44-3d16-4c10-b2ac-88eeeff554fd"}]: dispatch 2024-06-18T13:00:03.391 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:03 smithi167 ceph-mon[37701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81482f44-3d16-4c10-b2ac-88eeeff554fd"}]': finished 2024-06-18T13:00:03.391 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:03 smithi167 ceph-mon[37701]: osdmap e15: 3 total, 2 up, 3 in 2024-06-18T13:00:03.392 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:03 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:03.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:03 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4206571389' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81482f44-3d16-4c10-b2ac-88eeeff554fd"}]: dispatch 2024-06-18T13:00:03.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:03 smithi179 ceph-mon[33312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81482f44-3d16-4c10-b2ac-88eeeff554fd"}]: dispatch 2024-06-18T13:00:03.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:03 smithi179 ceph-mon[33312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81482f44-3d16-4c10-b2ac-88eeeff554fd"}]': finished 2024-06-18T13:00:03.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:03 smithi179 ceph-mon[33312]: osdmap e15: 3 total, 2 up, 3 in 2024-06-18T13:00:03.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:03 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:04.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:04 smithi167 ceph-mon[37701]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:04.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4184650799' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:00:04.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:04 smithi167 ceph-mon[29496]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:04.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4184650799' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:00:04.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:04 smithi179 ceph-mon[33312]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:04.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4184650799' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:00:05.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:05 smithi179 ceph-mon[33312]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:05.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:05 smithi167 ceph-mon[37701]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:05.888 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:05 smithi167 ceph-mon[29496]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:07.867 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:07 smithi167 ceph-mon[37701]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:07.867 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:07 smithi167 ceph-mon[29496]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:07.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:07 smithi179 ceph-mon[33312]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:09.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:09 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-18T13:00:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:09 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:09 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-18T13:00:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:09 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:09.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:09 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-18T13:00:09.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:09 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:10.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:10 smithi167 ceph-mon[29496]: Deploying daemon osd.2 on smithi167 2024-06-18T13:00:10.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:10 smithi167 ceph-mon[29496]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:10 smithi167 ceph-mon[37701]: Deploying daemon osd.2 on smithi167 2024-06-18T13:00:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:10 smithi167 ceph-mon[37701]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:10 smithi179 ceph-mon[33312]: Deploying daemon osd.2 on smithi167 2024-06-18T13:00:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:10 smithi179 ceph-mon[33312]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:11.707 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:11 smithi167 ceph-mon[37701]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:11.707 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:11 smithi167 ceph-mon[29496]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:11.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:11 smithi179 ceph-mon[33312]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:13.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:13 smithi167 ceph-mon[29496]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:13 smithi167 ceph-mon[37701]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:13.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:13 smithi179 ceph-mon[33312]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:14.848 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:14 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:00:14.849 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:14 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:14.849 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:14 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:14.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:14 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:00:14.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:14 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:14.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:14 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:14.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:14 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:00:14.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:14 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:14.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:14 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:15.741 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:15 smithi167 ceph-mon[29496]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:15.741 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:15 smithi167 ceph-mon[37701]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:15 smithi179 ceph-mon[33312]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:17.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:17.051+0000 7ff48bfff640 1 -- 172.21.15.167:0/3538136236 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff4a402d590 con 0x7ff48c06dec0 2024-06-18T13:00:17.051 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 2 on host 'smithi167' 2024-06-18T13:00:17.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:17.053+0000 7ff4a92c3640 1 -- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff48c06dec0 msgr2=0x7ff48c070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:00:17.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:17.053+0000 7ff4a92c3640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff48c06dec0 0x7ff48c070380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff49800bbe0 tx=0x7ff49800b390 comp rx=0 tx=0).stop 2024-06-18T13:00:17.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:17.053+0000 7ff4a92c3640 1 -- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4a41013c0 msgr2=0x7ff4a4196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:00:17.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:17.053+0000 7ff4a92c3640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4a41013c0 0x7ff4a4196c30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff494005940 tx=0x7ff494005dc0 comp rx=0 tx=0).stop 2024-06-18T13:00:17.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:17.053+0000 7ff4a92c3640 1 -- 172.21.15.167:0/3538136236 shutdown_connections 2024-06-18T13:00:17.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:17.053+0000 7ff4a92c3640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff48c06dec0 0x7ff48c070380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:17.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:17.053+0000 7ff4a92c3640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a4102f80 0x7ff4a419e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:17.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:17.053+0000 7ff4a92c3640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4a41025c0 0x7ff4a4197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:17.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:17.053+0000 7ff4a92c3640 1 --2- 172.21.15.167:0/3538136236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4a41013c0 0x7ff4a4196c30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:17.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:17.053+0000 7ff4a92c3640 1 -- 172.21.15.167:0/3538136236 >> 172.21.15.167:0/3538136236 conn(0x7ff4a40fcb70 msgr2=0x7ff4a40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:00:17.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:17.053+0000 7ff4a92c3640 1 -- 172.21.15.167:0/3538136236 shutdown_connections 2024-06-18T13:00:17.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:17.053+0000 7ff4a92c3640 1 -- 172.21.15.167:0/3538136236 wait complete. 2024-06-18T13:00:17.671 DEBUG:teuthology.orchestra.run.smithi167:osd.2> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.2.service 2024-06-18T13:00:17.673 INFO:tasks.cephadm:Deploying osd.3 on smithi167 with /dev/vg_nvme/lv_1... 2024-06-18T13:00:17.673 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-18T13:00:18.037 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:00:18.324 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:18 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:18.324 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:18 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:18.324 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:18 smithi167 ceph-mon[29496]: from='osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-18T13:00:18.324 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:18 smithi167 ceph-mon[29496]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:18.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:18 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:18.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:18 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:18.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:18 smithi167 ceph-mon[37701]: from='osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-18T13:00:18.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:18 smithi167 ceph-mon[37701]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:18.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:18 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:18.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:18 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:18.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:18 smithi179 ceph-mon[33312]: from='osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-18T13:00:18.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:18 smithi179 ceph-mon[33312]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:19.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:19 smithi167 ceph-mon[37701]: from='osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-18T13:00:19.363 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:19 smithi167 ceph-mon[37701]: osdmap e16: 3 total, 2 up, 3 in 2024-06-18T13:00:19.363 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:19 smithi167 ceph-mon[37701]: from='osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T13:00:19.363 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:19 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:19.363 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:19 smithi167 ceph-mon[29496]: from='osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-18T13:00:19.363 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:19 smithi167 ceph-mon[29496]: osdmap e16: 3 total, 2 up, 3 in 2024-06-18T13:00:19.363 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:19 smithi167 ceph-mon[29496]: from='osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T13:00:19.363 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:19 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:19.364 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:00:19 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2[52740]: 2024-06-18T13:00:19.066+0000 7f087ee4b640 -1 osd.2 0 waiting for initial osdmap 2024-06-18T13:00:19.364 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:00:19 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2[52740]: 2024-06-18T13:00:19.072+0000 7f087ac73640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-18T13:00:19.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:19 smithi179 ceph-mon[33312]: from='osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-18T13:00:19.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:19 smithi179 ceph-mon[33312]: osdmap e16: 3 total, 2 up, 3 in 2024-06-18T13:00:19.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:19 smithi179 ceph-mon[33312]: from='osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T13:00:19.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:19 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:19.813 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-18T13:00:19.899 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-18T13:00:19.917 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-06-18T13:00:19.917 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-06-18T13:00:19.918 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0171787 s, 610 MB/s 2024-06-18T13:00:19.918 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-06-18T13:00:20.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:20 smithi179 ceph-mon[33312]: from='osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-18T13:00:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:20 smithi179 ceph-mon[33312]: osdmap e17: 3 total, 2 up, 3 in 2024-06-18T13:00:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:20 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:20 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:20 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:20 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:20 smithi179 ceph-mon[33312]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:20.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:20 smithi167 ceph-mon[29496]: from='osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-18T13:00:20.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:20 smithi167 ceph-mon[29496]: osdmap e17: 3 total, 2 up, 3 in 2024-06-18T13:00:20.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:20 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:20.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:20 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:20.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:20 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:20.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:20 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:20.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:20 smithi167 ceph-mon[29496]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:20 smithi167 ceph-mon[37701]: from='osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-18T13:00:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:20 smithi167 ceph-mon[37701]: osdmap e17: 3 total, 2 up, 3 in 2024-06-18T13:00:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:20 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:20 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:20 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:20 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:20 smithi167 ceph-mon[37701]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-18T13:00:21.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:21 smithi167 ceph-mon[29496]: purged_snaps scrub starts 2024-06-18T13:00:21.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:21 smithi167 ceph-mon[29496]: purged_snaps scrub ok 2024-06-18T13:00:21.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:21 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:21.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:21 smithi167 ceph-mon[29496]: osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921] boot 2024-06-18T13:00:21.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:21 smithi167 ceph-mon[29496]: osdmap e18: 3 total, 3 up, 3 in 2024-06-18T13:00:21.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:21 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:21.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:21 smithi167 ceph-mon[37701]: purged_snaps scrub starts 2024-06-18T13:00:21.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:21 smithi167 ceph-mon[37701]: purged_snaps scrub ok 2024-06-18T13:00:21.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:21 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:21.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:21 smithi167 ceph-mon[37701]: osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921] boot 2024-06-18T13:00:21.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:21 smithi167 ceph-mon[37701]: osdmap e18: 3 total, 3 up, 3 in 2024-06-18T13:00:21.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:21 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:21 smithi179 ceph-mon[33312]: purged_snaps scrub starts 2024-06-18T13:00:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:21 smithi179 ceph-mon[33312]: purged_snaps scrub ok 2024-06-18T13:00:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:21 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:21 smithi179 ceph-mon[33312]: osd.2 [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921] boot 2024-06-18T13:00:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:21 smithi179 ceph-mon[33312]: osdmap e18: 3 total, 3 up, 3 in 2024-06-18T13:00:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:21 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:00:21.411 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_1 2024-06-18T13:00:21.788 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:00:22.426 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:22 smithi167 ceph-mon[29496]: osdmap e19: 3 total, 3 up, 3 in 2024-06-18T13:00:22.426 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:22 smithi167 ceph-mon[29496]: pgmap v73: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:22.426 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:22 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:00:22.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:22 smithi167 ceph-mon[37701]: osdmap e19: 3 total, 3 up, 3 in 2024-06-18T13:00:22.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:22 smithi167 ceph-mon[37701]: pgmap v73: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:22.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:22 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:00:22.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:22 smithi179 ceph-mon[33312]: osdmap e19: 3 total, 3 up, 3 in 2024-06-18T13:00:22.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:22 smithi179 ceph-mon[33312]: pgmap v73: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:22.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:22 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:00:23.373 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-18T13:00:23.373 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: osdmap e20: 3 total, 3 up, 3 in 2024-06-18T13:00:23.373 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:00:23.373 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.373 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:23.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:23.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:23.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:23.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:00:23.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:00:23.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:23.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:00:23.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.375 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-18T13:00:23.375 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: osdmap e20: 3 total, 3 up, 3 in 2024-06-18T13:00:23.375 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:00:23.375 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.375 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.375 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:23.375 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:23.375 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:23.375 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.375 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:23.375 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:00:23.375 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.375 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:00:23.376 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:23.376 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:00:23.376 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:23 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-18T13:00:23.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: osdmap e20: 3 total, 3 up, 3 in 2024-06-18T13:00:23.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:00:23.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:23.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:23.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:23.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:23.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:00:23.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:00:23.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:23.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:00:23.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:23 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:23.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.873+0000 7f7c6370f640 1 -- 172.21.15.167:0/1129079804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c5c101e70 msgr2=0x7f7c5c104260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:00:23.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.873+0000 7f7c6370f640 1 --2- 172.21.15.167:0/1129079804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c5c101e70 0x7f7c5c104260 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009900 tx=0x7f7c5802a3f0 comp rx=0 tx=0).stop 2024-06-18T13:00:23.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.873+0000 7f7c6370f640 1 -- 172.21.15.167:0/1129079804 shutdown_connections 2024-06-18T13:00:23.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.873+0000 7f7c6370f640 1 --2- 172.21.15.167:0/1129079804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c5c1070d0 0x7f7c5c1094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:23.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.873+0000 7f7c6370f640 1 --2- 172.21.15.167:0/1129079804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c5c1047a0 0x7f7c5c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:23.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.873+0000 7f7c6370f640 1 --2- 172.21.15.167:0/1129079804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c5c101e70 0x7f7c5c104260 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:23.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.873+0000 7f7c6370f640 1 -- 172.21.15.167:0/1129079804 >> 172.21.15.167:0/1129079804 conn(0x7f7c5c0fba30 msgr2=0x7f7c5c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:00:23.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.873+0000 7f7c6370f640 1 -- 172.21.15.167:0/1129079804 shutdown_connections 2024-06-18T13:00:23.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.873+0000 7f7c6370f640 1 -- 172.21.15.167:0/1129079804 wait complete. 2024-06-18T13:00:23.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.873+0000 7f7c6370f640 1 Processor -- start 2024-06-18T13:00:23.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.873+0000 7f7c6370f640 1 -- start start 2024-06-18T13:00:23.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.873+0000 7f7c6370f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c5c101e70 0x7f7c5c1017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:00:23.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.873+0000 7f7c6370f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c5c1047a0 0x7f7c5c0ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:00:23.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.873+0000 7f7c6370f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c5c1070d0 0x7f7c5c100360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:00:23.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c6370f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c10c4f0 con 0x7f7c5c101e70 2024-06-18T13:00:23.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c6370f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c10c370 con 0x7f7c5c1047a0 2024-06-18T13:00:23.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c6370f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5c10c670 con 0x7f7c5c1070d0 2024-06-18T13:00:23.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c614ab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c5c101e70 0x7f7c5c1017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:00:23.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c614ab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c5c101e70 0x7f7c5c1017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40870/0 (socket says 172.21.15.167:40870) 2024-06-18T13:00:23.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c614ab640 1 -- 172.21.15.167:0/3857068995 learned_addr learned my addr 172.21.15.167:0/3857068995 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:00:23.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c61cac640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c5c1070d0 0x7f7c5c100360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:00:23.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c60caa640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c5c1047a0 0x7f7c5c0ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:00:23.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c614ab640 1 -- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c5c1070d0 msgr2=0x7f7c5c100360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:00:23.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c614ab640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c5c1070d0 0x7f7c5c100360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:23.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c614ab640 1 -- 172.21.15.167:0/3857068995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c5c1047a0 msgr2=0x7f7c5c0ffe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:00:23.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c614ab640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c5c1047a0 0x7f7c5c0ffe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:23.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c614ab640 1 -- 172.21.15.167:0/3857068995 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5c100c20 con 0x7f7c5c101e70 2024-06-18T13:00:23.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c61cac640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c5c1070d0 0x7f7c5c100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:00:23.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c614ab640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c5c101e70 0x7f7c5c1017d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7c58005ab0 tx=0x7f7c580045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:00:23.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c527fc640 1 -- 172.21.15.167:0/3857068995 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5803f070 con 0x7f7c5c101e70 2024-06-18T13:00:23.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c527fc640 1 -- 172.21.15.167:0/3857068995 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f7c58003a70 con 0x7f7c5c101e70 2024-06-18T13:00:23.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c6370f640 1 -- 172.21.15.167:0/3857068995 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5c1a0d80 con 0x7f7c5c101e70 2024-06-18T13:00:23.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.874+0000 7f7c527fc640 1 -- 172.21.15.167:0/3857068995 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58035030 con 0x7f7c5c101e70 2024-06-18T13:00:23.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.875+0000 7f7c6370f640 1 -- 172.21.15.167:0/3857068995 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5c1a1180 con 0x7f7c5c101e70 2024-06-18T13:00:23.883 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:00:23 smithi167 sudo[56459]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-18T13:00:23.883 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:00:23 smithi167 sudo[56459]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-18T13:00:23.883 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:00:23 smithi167 sudo[56459]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-18T13:00:23.883 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:00:23 smithi167 sudo[56459]: pam_unix(sudo:session): session closed for user root 2024-06-18T13:00:23.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.876+0000 7f7c6370f640 1 -- 172.21.15.167:0/3857068995 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c5c104b80 con 0x7f7c5c101e70 2024-06-18T13:00:23.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.876+0000 7f7c527fc640 1 -- 172.21.15.167:0/3857068995 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7f7c58042070 con 0x7f7c5c101e70 2024-06-18T13:00:23.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.877+0000 7f7c527fc640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f7c4806dec0 0x7f7c48070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:00:23.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.877+0000 7f7c60caa640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f7c4806dec0 0x7f7c48070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:00:23.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.877+0000 7f7c527fc640 1 -- 172.21.15.167:0/3857068995 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3017+0+0 (secure 0 0 0) 0x7f7c580adea0 con 0x7f7c5c101e70 2024-06-18T13:00:23.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.877+0000 7f7c60caa640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f7c4806dec0 0x7f7c48070380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0095f0 tx=0x7f7c4c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:00:23.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:23.884+0000 7f7c527fc640 1 -- 172.21.15.167:0/3857068995 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f7c5807be60 con 0x7f7c5c101e70 2024-06-18T13:00:24.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:24.068+0000 7f7c6370f640 1 -- 172.21.15.167:0/3857068995 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7c5c061de0 con 0x7f7c4806dec0 2024-06-18T13:00:24.194 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56485]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-18T13:00:24.195 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56485]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-18T13:00:24.195 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56485]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-18T13:00:24.195 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56485]: pam_unix(sudo:session): session closed for user root 2024-06-18T13:00:24.476 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56593]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-18T13:00:24.476 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56593]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-18T13:00:24.477 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56593]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-18T13:00:24.477 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56593]: pam_unix(sudo:session): session closed for user root 2024-06-18T13:00:24.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[37701]: Detected new or changed devices on smithi167 2024-06-18T13:00:24.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[37701]: Adjusting osd_memory_target on smithi167 to 5354M 2024-06-18T13:00:24.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-18T13:00:24.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[37701]: osdmap e21: 3 total, 3 up, 3 in 2024-06-18T13:00:24.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[37701]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:24.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:00:24.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:00:24.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:24.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:24.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[29496]: Detected new or changed devices on smithi167 2024-06-18T13:00:24.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[29496]: Adjusting osd_memory_target on smithi167 to 5354M 2024-06-18T13:00:24.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-18T13:00:24.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[29496]: osdmap e21: 3 total, 3 up, 3 in 2024-06-18T13:00:24.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[29496]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:24.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:00:24.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:00:24.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:24.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:24 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:24.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:24 smithi179 ceph-mon[33312]: Detected new or changed devices on smithi167 2024-06-18T13:00:24.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:24 smithi179 ceph-mon[33312]: Adjusting osd_memory_target on smithi167 to 5354M 2024-06-18T13:00:24.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:24 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-18T13:00:24.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:24 smithi179 ceph-mon[33312]: osdmap e21: 3 total, 3 up, 3 in 2024-06-18T13:00:24.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:24 smithi179 ceph-mon[33312]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:24.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:24 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:00:24.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:24 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:00:24.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:24 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:24.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:24 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:24.797 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56613]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-18T13:00:24.798 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56613]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-18T13:00:24.798 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56613]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-18T13:00:24.798 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56613]: pam_unix(sudo:session): session closed for user root 2024-06-18T13:00:25.148 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56628]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-18T13:00:25.148 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56628]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-18T13:00:25.148 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56628]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-18T13:00:25.148 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:00:24 smithi167 sudo[56628]: pam_unix(sudo:session): session closed for user root 2024-06-18T13:00:25.411 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:25 smithi167 sudo[56640]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-18T13:00:25.411 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:25 smithi167 sudo[56640]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-18T13:00:25.412 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:25 smithi167 sudo[56640]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-18T13:00:25.412 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:00:25 smithi167 sudo[56633]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-18T13:00:25.412 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:00:25 smithi167 sudo[56633]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-18T13:00:25.412 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:00:25 smithi167 sudo[56633]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-18T13:00:25.412 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:00:25 smithi167 sudo[56633]: pam_unix(sudo:session): session closed for user root 2024-06-18T13:00:25.663 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:25 smithi167 sudo[56640]: pam_unix(sudo:session): session closed for user root 2024-06-18T13:00:25.925 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:25 smithi179 ceph-mon[33312]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:00:25.925 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:25 smithi179 ceph-mon[33312]: osdmap e22: 3 total, 3 up, 3 in 2024-06-18T13:00:25.925 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:25 smithi179 ceph-mon[33312]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-18T13:00:25.925 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:25 smithi179 sudo[35804]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-18T13:00:25.926 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:25 smithi179 sudo[35804]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-18T13:00:25.926 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:25 smithi179 sudo[35804]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-18T13:00:25.926 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:25 smithi179 sudo[35804]: pam_unix(sudo:session): session closed for user root 2024-06-18T13:00:25.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:25 smithi167 ceph-mon[37701]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:00:25.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:25 smithi167 ceph-mon[37701]: osdmap e22: 3 total, 3 up, 3 in 2024-06-18T13:00:25.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:25 smithi167 ceph-mon[37701]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-18T13:00:25.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:25 smithi167 sudo[56663]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-18T13:00:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:25 smithi167 sudo[56663]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-18T13:00:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:25 smithi167 sudo[56663]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-18T13:00:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:25 smithi167 sudo[56663]: pam_unix(sudo:session): session closed for user root 2024-06-18T13:00:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:25 smithi167 ceph-mon[29496]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:00:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:25 smithi167 ceph-mon[29496]: osdmap e22: 3 total, 3 up, 3 in 2024-06-18T13:00:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:25 smithi167 ceph-mon[29496]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-18T13:00:26.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:26.175+0000 7f7c527fc640 1 -- 172.21.15.167:0/3857068995 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mgrmap(e 15) ==== 92117+0+0 (secure 0 0 0) 0x7f7c580b1030 con 0x7f7c5c101e70 2024-06-18T13:00:26.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 sudo[35807]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-18T13:00:26.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 sudo[35807]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-18T13:00:26.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 sudo[35807]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-18T13:00:26.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 sudo[35807]: pam_unix(sudo:session): session closed for user root 2024-06-18T13:00:26.704 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 sudo[56713]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-18T13:00:26.704 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 sudo[56713]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-18T13:00:26.704 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 sudo[56713]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-18T13:00:26.704 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 sudo[56713]: pam_unix(sudo:session): session closed for user root 2024-06-18T13:00:26.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:26.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-18T13:00:26.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T13:00:26.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T13:00:26.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T13:00:26.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-18T13:00:26.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T13:00:26.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T13:00:26.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T13:00:26.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-18T13:00:26.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: mgrmap e15: y(active, since 2m), standbys: x 2024-06-18T13:00:26.963 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:26.964 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T13:00:26.964 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T13:00:26.964 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T13:00:26.964 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2888763117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5958775e-faaa-4fba-9b33-dcd46c049926"}]: dispatch 2024-06-18T13:00:26.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 sudo[56717]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-18T13:00:26.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 sudo[56717]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-18T13:00:26.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 sudo[56717]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-18T13:00:26.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 sudo[56717]: pam_unix(sudo:session): session closed for user root 2024-06-18T13:00:26.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:26.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-18T13:00:26.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T13:00:26.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T13:00:26.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T13:00:26.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-18T13:00:26.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T13:00:26.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T13:00:26.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T13:00:26.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-18T13:00:26.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: mgrmap e15: y(active, since 2m), standbys: x 2024-06-18T13:00:26.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:26.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T13:00:26.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T13:00:26.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T13:00:26.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2888763117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5958775e-faaa-4fba-9b33-dcd46c049926"}]: dispatch 2024-06-18T13:00:27.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:27.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-18T13:00:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T13:00:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T13:00:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T13:00:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-18T13:00:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T13:00:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T13:00:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T13:00:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-18T13:00:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: mgrmap e15: y(active, since 2m), standbys: x 2024-06-18T13:00:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T13:00:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T13:00:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T13:00:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2888763117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5958775e-faaa-4fba-9b33-dcd46c049926"}]: dispatch 2024-06-18T13:00:27.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:27 smithi167 ceph-mon[29496]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-18T13:00:27.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:27 smithi167 ceph-mon[29496]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-18T13:00:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2888763117' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5958775e-faaa-4fba-9b33-dcd46c049926"}]': finished 2024-06-18T13:00:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:27 smithi167 ceph-mon[29496]: osdmap e23: 4 total, 3 up, 4 in 2024-06-18T13:00:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:27 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1291974365' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:00:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:27 smithi167 ceph-mon[29496]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:27 smithi167 ceph-mon[37701]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-18T13:00:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:27 smithi167 ceph-mon[37701]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-18T13:00:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2888763117' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5958775e-faaa-4fba-9b33-dcd46c049926"}]': finished 2024-06-18T13:00:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:27 smithi167 ceph-mon[37701]: osdmap e23: 4 total, 3 up, 4 in 2024-06-18T13:00:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:27 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1291974365' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:00:27.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:27 smithi167 ceph-mon[37701]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:28.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:27 smithi179 ceph-mon[33312]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-18T13:00:28.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:27 smithi179 ceph-mon[33312]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-18T13:00:28.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2888763117' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5958775e-faaa-4fba-9b33-dcd46c049926"}]': finished 2024-06-18T13:00:28.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:27 smithi179 ceph-mon[33312]: osdmap e23: 4 total, 3 up, 4 in 2024-06-18T13:00:28.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:27 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:28.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1291974365' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:00:28.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:27 smithi179 ceph-mon[33312]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:29.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:29 smithi167 ceph-mon[29496]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:29 smithi167 ceph-mon[37701]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:29.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:29 smithi179 ceph-mon[33312]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:31.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:31 smithi167 ceph-mon[29496]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:31 smithi167 ceph-mon[37701]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:31.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:31 smithi179 ceph-mon[33312]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:33.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:32 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-18T13:00:33.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:32 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:33.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:32 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-18T13:00:33.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:32 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:33.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:32 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-18T13:00:33.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:32 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:34.135 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:33 smithi167 ceph-mon[29496]: Deploying daemon osd.3 on smithi167 2024-06-18T13:00:34.135 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:33 smithi167 ceph-mon[29496]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 39 KiB/s wr, 3 op/s 2024-06-18T13:00:34.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:33 smithi167 ceph-mon[37701]: Deploying daemon osd.3 on smithi167 2024-06-18T13:00:34.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:33 smithi167 ceph-mon[37701]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 39 KiB/s wr, 3 op/s 2024-06-18T13:00:34.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:33 smithi179 ceph-mon[33312]: Deploying daemon osd.3 on smithi167 2024-06-18T13:00:34.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:33 smithi179 ceph-mon[33312]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 39 KiB/s wr, 3 op/s 2024-06-18T13:00:35.688 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:35 smithi167 ceph-mon[37701]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 30 KiB/s wr, 2 op/s 2024-06-18T13:00:35.689 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:35 smithi167 ceph-mon[29496]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 30 KiB/s wr, 2 op/s 2024-06-18T13:00:35.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:35 smithi179 ceph-mon[33312]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 30 KiB/s wr, 2 op/s 2024-06-18T13:00:37.860 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:37 smithi167 ceph-mon[29496]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 24 KiB/s wr, 2 op/s 2024-06-18T13:00:37.860 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:37 smithi167 ceph-mon[37701]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 24 KiB/s wr, 2 op/s 2024-06-18T13:00:37.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:37 smithi179 ceph-mon[33312]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 24 KiB/s wr, 2 op/s 2024-06-18T13:00:38.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:38 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:00:38.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:38 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:38.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:38 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:38.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:38 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:00:38.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:38 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:38.827 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:38 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:38.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:38 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:00:38.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:38 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:38.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:38 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:39.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:39 smithi167 ceph-mon[29496]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-18T13:00:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:39 smithi167 ceph-mon[37701]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-18T13:00:39.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:39 smithi179 ceph-mon[33312]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-18T13:00:41.499 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 3 on host 'smithi167' 2024-06-18T13:00:41.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:41.498+0000 7f7c527fc640 1 -- 172.21.15.167:0/3857068995 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7c5c061de0 con 0x7f7c4806dec0 2024-06-18T13:00:41.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:41.500+0000 7f7c6370f640 1 -- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f7c4806dec0 msgr2=0x7f7c48070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:00:41.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:41.500+0000 7f7c6370f640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f7c4806dec0 0x7f7c48070380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0095f0 tx=0x7f7c4c005f90 comp rx=0 tx=0).stop 2024-06-18T13:00:41.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:41.501+0000 7f7c6370f640 1 -- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c5c101e70 msgr2=0x7f7c5c1017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:00:41.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:41.501+0000 7f7c6370f640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c5c101e70 0x7f7c5c1017d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7c58005ab0 tx=0x7f7c580045c0 comp rx=0 tx=0).stop 2024-06-18T13:00:41.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:41.501+0000 7f7c6370f640 1 -- 172.21.15.167:0/3857068995 shutdown_connections 2024-06-18T13:00:41.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:41.501+0000 7f7c6370f640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f7c4806dec0 0x7f7c48070380 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:41.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:41.501+0000 7f7c6370f640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c5c1070d0 0x7f7c5c100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:41.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:41.501+0000 7f7c6370f640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c5c1047a0 0x7f7c5c0ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:41.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:41.501+0000 7f7c6370f640 1 --2- 172.21.15.167:0/3857068995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c5c101e70 0x7f7c5c1017d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:41.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:41.501+0000 7f7c6370f640 1 -- 172.21.15.167:0/3857068995 >> 172.21.15.167:0/3857068995 conn(0x7f7c5c0fba30 msgr2=0x7f7c5c102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:00:41.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:41.501+0000 7f7c6370f640 1 -- 172.21.15.167:0/3857068995 shutdown_connections 2024-06-18T13:00:41.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:00:41.501+0000 7f7c6370f640 1 -- 172.21.15.167:0/3857068995 wait complete. 2024-06-18T13:00:42.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:41 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:41 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:41 smithi179 ceph-mon[33312]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-18T13:00:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:41 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:41 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:00:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:41 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:41 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:00:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:41 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:41 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:41 smithi179 ceph-mon[33312]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-18T13:00:42.188 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[37701]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-18T13:00:42.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:42.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:00:42.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:00:42.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[37701]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-18T13:00:42.190 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.190 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.190 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[29496]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-18T13:00:42.190 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:42.190 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:00:42.190 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.190 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:00:42.190 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.190 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:42.190 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:41 smithi167 ceph-mon[29496]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-18T13:00:42.216 DEBUG:teuthology.orchestra.run.smithi167:osd.3> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.3.service 2024-06-18T13:00:42.218 INFO:tasks.cephadm:Deploying osd.4 on smithi179 with /dev/vg_nvme/lv_4... 2024-06-18T13:00:42.218 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-18T13:00:42.464 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:00:43.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:42 smithi179 ceph-mon[33312]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-18T13:00:43.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:42 smithi179 ceph-mon[33312]: osdmap e24: 4 total, 3 up, 4 in 2024-06-18T13:00:43.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:42 smithi179 ceph-mon[33312]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T13:00:43.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:42 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:43.200 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:42 smithi167 ceph-mon[29496]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-18T13:00:43.200 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:42 smithi167 ceph-mon[29496]: osdmap e24: 4 total, 3 up, 4 in 2024-06-18T13:00:43.201 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:42 smithi167 ceph-mon[29496]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T13:00:43.201 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:42 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:43.201 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:42 smithi167 ceph-mon[37701]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-18T13:00:43.201 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:42 smithi167 ceph-mon[37701]: osdmap e24: 4 total, 3 up, 4 in 2024-06-18T13:00:43.201 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:42 smithi167 ceph-mon[37701]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-18T13:00:43.201 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:42 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:43.464 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3[58367]: 2024-06-18T13:00:43.200+0000 7f893403a640 -1 osd.3 0 waiting for initial osdmap 2024-06-18T13:00:43.464 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3[58367]: 2024-06-18T13:00:43.205+0000 7f892be5a640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-18T13:00:43.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:43 smithi179 ceph-mon[33312]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-18T13:00:43.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:43 smithi179 ceph-mon[33312]: osdmap e25: 4 total, 3 up, 4 in 2024-06-18T13:00:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:43 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:43 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:43 smithi179 ceph-mon[33312]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' 2024-06-18T13:00:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:43 smithi179 ceph-mon[33312]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:43 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:43.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-mon[37701]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-18T13:00:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-mon[37701]: osdmap e25: 4 total, 3 up, 4 in 2024-06-18T13:00:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-mon[37701]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' 2024-06-18T13:00:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-mon[37701]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-mon[29496]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-18T13:00:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-mon[29496]: osdmap e25: 4 total, 3 up, 4 in 2024-06-18T13:00:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-mon[29496]: from='osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547]' entity='osd.3' 2024-06-18T13:00:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-mon[29496]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-18T13:00:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:43 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:44.122 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-18T13:00:44.211 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-18T13:00:44.226 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-06-18T13:00:44.227 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-06-18T13:00:44.227 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133819 s, 784 MB/s 2024-06-18T13:00:44.227 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-06-18T13:00:44.833 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_4 2024-06-18T13:00:45.087 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:00:45.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: purged_snaps scrub starts 2024-06-18T13:00:45.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: purged_snaps scrub ok 2024-06-18T13:00:45.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547] boot 2024-06-18T13:00:45.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: osdmap e26: 4 total, 4 up, 4 in 2024-06-18T13:00:45.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:45.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:45.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:45.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:45.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:45.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:45.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:45.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:45.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:45.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:00:45.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:45.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: purged_snaps scrub starts 2024-06-18T13:00:45.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: purged_snaps scrub ok 2024-06-18T13:00:45.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547] boot 2024-06-18T13:00:45.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: osdmap e26: 4 total, 4 up, 4 in 2024-06-18T13:00:45.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:45.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:45.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:45.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:45.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:45.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:45.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:45.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:45.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:45.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:00:45.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:45 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:45.516 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: purged_snaps scrub starts 2024-06-18T13:00:45.516 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: purged_snaps scrub ok 2024-06-18T13:00:45.516 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: osd.3 [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547] boot 2024-06-18T13:00:45.517 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: osdmap e26: 4 total, 4 up, 4 in 2024-06-18T13:00:45.517 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:00:45.517 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:45.517 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:45.517 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:45.517 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:45.517 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:45.517 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:00:45.517 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:45.517 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:45.517 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:00:45.517 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:45 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:00:46.435 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:46 smithi179 ceph-mon[33312]: Detected new or changed devices on smithi167 2024-06-18T13:00:46.435 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:46 smithi179 ceph-mon[33312]: Adjusting osd_memory_target on smithi167 to 4016M 2024-06-18T13:00:46.435 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:46 smithi179 ceph-mon[33312]: osdmap e27: 4 total, 4 up, 4 in 2024-06-18T13:00:46.435 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:46 smithi179 ceph-mon[33312]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:46.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:46 smithi167 ceph-mon[37701]: Detected new or changed devices on smithi167 2024-06-18T13:00:46.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:46 smithi167 ceph-mon[37701]: Adjusting osd_memory_target on smithi167 to 4016M 2024-06-18T13:00:46.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:46 smithi167 ceph-mon[37701]: osdmap e27: 4 total, 4 up, 4 in 2024-06-18T13:00:46.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:46 smithi167 ceph-mon[37701]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:46.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:46 smithi167 ceph-mon[29496]: Detected new or changed devices on smithi167 2024-06-18T13:00:46.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:46 smithi167 ceph-mon[29496]: Adjusting osd_memory_target on smithi167 to 4016M 2024-06-18T13:00:46.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:46 smithi167 ceph-mon[29496]: osdmap e27: 4 total, 4 up, 4 in 2024-06-18T13:00:46.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:46 smithi167 ceph-mon[29496]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:46.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.623+0000 7fade40d5640 1 -- 172.21.15.179:0/881701570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faddc102f70 msgr2=0x7faddc109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:00:46.625 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.623+0000 7fade40d5640 1 --2- 172.21.15.179:0/881701570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faddc102f70 0x7faddc109800 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009ca0 tx=0x7fadd402a8c0 comp rx=0 tx=0).stop 2024-06-18T13:00:46.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.623+0000 7fade40d5640 1 -- 172.21.15.179:0/881701570 shutdown_connections 2024-06-18T13:00:46.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.623+0000 7fade40d5640 1 --2- 172.21.15.179:0/881701570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faddc102f70 0x7faddc109800 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:46.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.623+0000 7fade40d5640 1 --2- 172.21.15.179:0/881701570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faddc1025b0 0x7faddc102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:46.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.623+0000 7fade40d5640 1 --2- 172.21.15.179:0/881701570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faddc1013b0 0x7faddc1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:46.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.623+0000 7fade40d5640 1 -- 172.21.15.179:0/881701570 >> 172.21.15.179:0/881701570 conn(0x7faddc0fcb40 msgr2=0x7faddc0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:00:46.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.623+0000 7fade40d5640 1 -- 172.21.15.179:0/881701570 shutdown_connections 2024-06-18T13:00:46.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.623+0000 7fade40d5640 1 -- 172.21.15.179:0/881701570 wait complete. 2024-06-18T13:00:46.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade40d5640 1 Processor -- start 2024-06-18T13:00:46.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade40d5640 1 -- start start 2024-06-18T13:00:46.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade40d5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faddc1013b0 0x7faddc196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:00:46.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade40d5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faddc1025b0 0x7faddc197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:00:46.626 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade40d5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faddc102f70 0x7faddc19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:00:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade40d5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faddc10c4e0 con 0x7faddc102f70 2024-06-18T13:00:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade40d5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faddc10c360 con 0x7faddc1013b0 2024-06-18T13:00:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade40d5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7faddc10c660 con 0x7faddc1025b0 2024-06-18T13:00:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade1e71640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faddc1013b0 0x7faddc196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:00:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade1e71640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faddc1013b0 0x7faddc196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:58922/0 (socket says 172.21.15.179:58922) 2024-06-18T13:00:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade1e71640 1 -- 172.21.15.179:0/2794396213 learned_addr learned my addr 172.21.15.179:0/2794396213 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T13:00:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade1670640 1 --2- 172.21.15.179:0/2794396213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faddc1025b0 0x7faddc197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:00:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade2672640 1 --2- 172.21.15.179:0/2794396213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faddc102f70 0x7faddc19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:00:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade1e71640 1 -- 172.21.15.179:0/2794396213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faddc1025b0 msgr2=0x7faddc197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:00:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade1e71640 1 --2- 172.21.15.179:0/2794396213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faddc1025b0 0x7faddc197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade1e71640 1 -- 172.21.15.179:0/2794396213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faddc102f70 msgr2=0x7faddc19e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:00:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade1e71640 1 --2- 172.21.15.179:0/2794396213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faddc102f70 0x7faddc19e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:00:46.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.624+0000 7fade1e71640 1 -- 172.21.15.179:0/2794396213 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc19e780 con 0x7faddc1013b0 2024-06-18T13:00:46.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.625+0000 7fade1e71640 1 --2- 172.21.15.179:0/2794396213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faddc1013b0 0x7faddc196b50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fadd800d770 tx=0x7fadd800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:00:46.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.625+0000 7fadd2ffd640 1 -- 172.21.15.179:0/2794396213 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8017070 con 0x7faddc1013b0 2024-06-18T13:00:46.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.625+0000 7fadd2ffd640 1 -- 172.21.15.179:0/2794396213 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fadd8009c30 con 0x7faddc1013b0 2024-06-18T13:00:46.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.625+0000 7fade40d5640 1 -- 172.21.15.179:0/2794396213 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc19ea70 con 0x7faddc1013b0 2024-06-18T13:00:46.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.625+0000 7fadd2ffd640 1 -- 172.21.15.179:0/2794396213 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8003b80 con 0x7faddc1013b0 2024-06-18T13:00:46.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.625+0000 7fade40d5640 1 -- 172.21.15.179:0/2794396213 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc19efb0 con 0x7faddc1013b0 2024-06-18T13:00:46.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.626+0000 7fade40d5640 1 -- 172.21.15.179:0/2794396213 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faddc1017b0 con 0x7faddc1013b0 2024-06-18T13:00:46.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.626+0000 7fadd2ffd640 1 -- 172.21.15.179:0/2794396213 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) ==== 92117+0+0 (secure 0 0 0) 0x7fadd8003d20 con 0x7faddc1013b0 2024-06-18T13:00:46.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.627+0000 7fadd2ffd640 1 --2- 172.21.15.179:0/2794396213 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fadc806df80 0x7fadc8070440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:00:46.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.627+0000 7fadd2ffd640 1 -- 172.21.15.179:0/2794396213 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3449+0+0 (secure 0 0 0) 0x7fadd808cef0 con 0x7faddc1013b0 2024-06-18T13:00:46.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.627+0000 7fade1670640 1 --2- 172.21.15.179:0/2794396213 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fadc806df80 0x7fadc8070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:00:46.629 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.627+0000 7fade1670640 1 --2- 172.21.15.179:0/2794396213 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fadc806df80 0x7fadc8070440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faddc198070 tx=0x7fadcc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:00:46.631 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.632+0000 7fadd2ffd640 1 -- 172.21.15.179:0/2794396213 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fadd805ac20 con 0x7faddc1013b0 2024-06-18T13:00:46.813 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:00:46.813+0000 7fade40d5640 1 -- 172.21.15.179:0/2794396213 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7faddc061de0 con 0x7fadc806df80 2024-06-18T13:00:47.475 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:47 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:00:47.475 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:47 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:00:47.475 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:47 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:47.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:47 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:00:47.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:47 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:00:47.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:47 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:47.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:47 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:00:47.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:47 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:00:47.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:47 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:48.448 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:48 smithi179 ceph-mon[33312]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:00:48.448 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:48 smithi179 ceph-mon[33312]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:48.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:48 smithi167 ceph-mon[37701]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:00:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:48 smithi167 ceph-mon[37701]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:48 smithi167 ceph-mon[29496]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:00:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:48 smithi167 ceph-mon[29496]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:49.560 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.179:0/3643495436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6abe43b9-eb57-4e7c-afce-fa7ff1b829c3"}]: dispatch 2024-06-18T13:00:49.560 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6abe43b9-eb57-4e7c-afce-fa7ff1b829c3"}]: dispatch 2024-06-18T13:00:49.560 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6abe43b9-eb57-4e7c-afce-fa7ff1b829c3"}]': finished 2024-06-18T13:00:49.560 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:49 smithi179 ceph-mon[33312]: osdmap e28: 5 total, 4 up, 5 in 2024-06-18T13:00:49.560 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:49 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:00:49.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.179:0/3643495436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6abe43b9-eb57-4e7c-afce-fa7ff1b829c3"}]: dispatch 2024-06-18T13:00:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6abe43b9-eb57-4e7c-afce-fa7ff1b829c3"}]: dispatch 2024-06-18T13:00:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6abe43b9-eb57-4e7c-afce-fa7ff1b829c3"}]': finished 2024-06-18T13:00:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:49 smithi167 ceph-mon[29496]: osdmap e28: 5 total, 4 up, 5 in 2024-06-18T13:00:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:49 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:00:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.179:0/3643495436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6abe43b9-eb57-4e7c-afce-fa7ff1b829c3"}]: dispatch 2024-06-18T13:00:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6abe43b9-eb57-4e7c-afce-fa7ff1b829c3"}]: dispatch 2024-06-18T13:00:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6abe43b9-eb57-4e7c-afce-fa7ff1b829c3"}]': finished 2024-06-18T13:00:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:49 smithi167 ceph-mon[37701]: osdmap e28: 5 total, 4 up, 5 in 2024-06-18T13:00:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:49 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:00:50.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:50 smithi179 ceph-mon[33312]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.179:0/3012929834' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:00:50.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:50 smithi167 ceph-mon[29496]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:50.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.179:0/3012929834' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:00:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:50 smithi167 ceph-mon[37701]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.179:0/3012929834' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:00:51.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:51 smithi179 ceph-mon[33312]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:51 smithi167 ceph-mon[29496]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:51 smithi167 ceph-mon[37701]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:53.685 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:53 smithi179 ceph-mon[33312]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:53.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:53 smithi167 ceph-mon[29496]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:53 smithi167 ceph-mon[37701]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:55.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-18T13:00:55.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:54 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:55.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:54 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-18T13:00:55.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:54 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:55.275 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:54 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-18T13:00:55.275 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:54 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:00:56.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:55 smithi179 ceph-mon[33312]: Deploying daemon osd.4 on smithi179 2024-06-18T13:00:56.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:55 smithi179 ceph-mon[33312]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:56.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:55 smithi167 ceph-mon[29496]: Deploying daemon osd.4 on smithi179 2024-06-18T13:00:56.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:55 smithi167 ceph-mon[29496]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:56.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:55 smithi167 ceph-mon[37701]: Deploying daemon osd.4 on smithi179 2024-06-18T13:00:56.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:55 smithi167 ceph-mon[37701]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:57.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:57 smithi179 ceph-mon[33312]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:57.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:57 smithi167 ceph-mon[29496]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:57 smithi167 ceph-mon[37701]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:59.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:00:59 smithi179 ceph-mon[33312]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:59.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:00:59 smithi167 ceph-mon[29496]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:00:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:00:59 smithi167 ceph-mon[37701]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:01:00.879 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:00 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:00.879 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:00 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:00.879 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:00 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:00.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:00 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:00.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:00 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:00 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:00 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:00 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:00 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:01.780 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:01 smithi179 ceph-mon[33312]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:01:01.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:01 smithi167 ceph-mon[37701]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:01:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:01 smithi167 ceph-mon[29496]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:01:02.873 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:02.874 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:02.874 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:02.874 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:02.874 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:02.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:02.913+0000 7fadd2ffd640 1 -- 172.21.15.179:0/2794396213 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faddc061de0 con 0x7fadc806df80 2024-06-18T13:01:02.913 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 4 on host 'smithi179' 2024-06-18T13:01:02.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:02.916+0000 7fade40d5640 1 -- 172.21.15.179:0/2794396213 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fadc806df80 msgr2=0x7fadc8070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:02.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:02.916+0000 7fade40d5640 1 --2- 172.21.15.179:0/2794396213 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fadc806df80 0x7fadc8070440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faddc198070 tx=0x7fadcc00b390 comp rx=0 tx=0).stop 2024-06-18T13:01:02.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:02.916+0000 7fade40d5640 1 -- 172.21.15.179:0/2794396213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faddc1013b0 msgr2=0x7faddc196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:02.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:02.916+0000 7fade40d5640 1 --2- 172.21.15.179:0/2794396213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faddc1013b0 0x7faddc196b50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fadd800d770 tx=0x7fadd800dc30 comp rx=0 tx=0).stop 2024-06-18T13:01:02.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:02.916+0000 7fade40d5640 1 -- 172.21.15.179:0/2794396213 shutdown_connections 2024-06-18T13:01:02.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:02.916+0000 7fade40d5640 1 --2- 172.21.15.179:0/2794396213 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fadc806df80 0x7fadc8070440 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:02.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:02.916+0000 7fade40d5640 1 --2- 172.21.15.179:0/2794396213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faddc102f70 0x7faddc19e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:02.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:02.916+0000 7fade40d5640 1 --2- 172.21.15.179:0/2794396213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faddc1025b0 0x7faddc197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:02.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:02.916+0000 7fade40d5640 1 --2- 172.21.15.179:0/2794396213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faddc1013b0 0x7faddc196b50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:02.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:02.916+0000 7fade40d5640 1 -- 172.21.15.179:0/2794396213 >> 172.21.15.179:0/2794396213 conn(0x7faddc0fcb40 msgr2=0x7faddc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:01:02.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:02.916+0000 7fade40d5640 1 -- 172.21.15.179:0/2794396213 shutdown_connections 2024-06-18T13:01:02.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:02.916+0000 7fade40d5640 1 -- 172.21.15.179:0/2794396213 wait complete. 2024-06-18T13:01:02.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:02.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:03.460 DEBUG:teuthology.orchestra.run.smithi179:osd.4> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.4.service 2024-06-18T13:01:03.463 INFO:tasks.cephadm:Deploying osd.5 on smithi179 with /dev/vg_nvme/lv_3... 2024-06-18T13:01:03.463 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-18T13:01:03.813 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:03 smithi179 ceph-mon[33312]: from='osd.4 [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-18T13:01:03.813 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:03 smithi179 ceph-mon[33312]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-18T13:01:03.813 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:03 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:03.813 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:03 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:03.813 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:03 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:03.813 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:03 smithi179 ceph-mon[33312]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:01:03.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:03 smithi167 ceph-mon[29496]: from='osd.4 [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-18T13:01:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:03 smithi167 ceph-mon[29496]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-18T13:01:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:03 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:03 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:03 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:03 smithi167 ceph-mon[29496]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:01:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:03 smithi167 ceph-mon[37701]: from='osd.4 [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-18T13:01:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:03 smithi167 ceph-mon[37701]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-18T13:01:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:03 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:03 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:03 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:03 smithi167 ceph-mon[37701]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:01:04.108 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:01:04.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:04 smithi179 ceph-mon[33312]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-18T13:01:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:04 smithi179 ceph-mon[33312]: osdmap e29: 5 total, 4 up, 5 in 2024-06-18T13:01:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:04 smithi179 ceph-mon[33312]: from='osd.4 [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:04 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:01:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:04 smithi179 ceph-mon[33312]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:04 smithi179 ceph-mon[33312]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-18T13:01:04.887 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:01:04 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:01:04.620+0000 7f3ac097c640 -1 osd.4 0 waiting for initial osdmap 2024-06-18T13:01:04.887 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:01:04 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:01:04.625+0000 7f3abc7a4640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-18T13:01:04.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:04 smithi167 ceph-mon[37701]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-18T13:01:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:04 smithi167 ceph-mon[37701]: osdmap e29: 5 total, 4 up, 5 in 2024-06-18T13:01:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:04 smithi167 ceph-mon[37701]: from='osd.4 [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:04 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:01:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:04 smithi167 ceph-mon[37701]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:04 smithi167 ceph-mon[37701]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-18T13:01:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:04 smithi167 ceph-mon[29496]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-18T13:01:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:04 smithi167 ceph-mon[29496]: osdmap e29: 5 total, 4 up, 5 in 2024-06-18T13:01:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:04 smithi167 ceph-mon[29496]: from='osd.4 [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:04 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:01:04.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:04 smithi167 ceph-mon[29496]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:04.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:04 smithi167 ceph-mon[29496]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-18T13:01:05.771 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:05 smithi179 ceph-mon[33312]: osdmap e30: 5 total, 4 up, 5 in 2024-06-18T13:01:05.771 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:05 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:01:05.772 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:05 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:01:05.772 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:05 smithi179 ceph-mon[33312]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:01:05.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:05 smithi167 ceph-mon[37701]: osdmap e30: 5 total, 4 up, 5 in 2024-06-18T13:01:05.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:05 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:01:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:05 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:01:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:05 smithi167 ceph-mon[37701]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:01:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:05 smithi167 ceph-mon[29496]: osdmap e30: 5 total, 4 up, 5 in 2024-06-18T13:01:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:05 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:01:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:05 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:01:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:05 smithi167 ceph-mon[29496]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-18T13:01:06.560 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-18T13:01:06.640 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-18T13:01:06.654 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-06-18T13:01:06.654 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-06-18T13:01:06.654 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124302 s, 844 MB/s 2024-06-18T13:01:06.654 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-06-18T13:01:06.774 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:06 smithi179 ceph-mon[33312]: purged_snaps scrub starts 2024-06-18T13:01:06.774 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:06 smithi179 ceph-mon[33312]: purged_snaps scrub ok 2024-06-18T13:01:06.774 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:06 smithi179 ceph-mon[33312]: osd.4 [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696] boot 2024-06-18T13:01:06.774 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:06 smithi179 ceph-mon[33312]: osdmap e31: 5 total, 5 up, 5 in 2024-06-18T13:01:06.774 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:06 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:01:06.775 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:06 smithi179 ceph-mon[33312]: osdmap e32: 5 total, 5 up, 5 in 2024-06-18T13:01:06.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:06 smithi167 ceph-mon[37701]: purged_snaps scrub starts 2024-06-18T13:01:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:06 smithi167 ceph-mon[37701]: purged_snaps scrub ok 2024-06-18T13:01:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:06 smithi167 ceph-mon[37701]: osd.4 [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696] boot 2024-06-18T13:01:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:06 smithi167 ceph-mon[37701]: osdmap e31: 5 total, 5 up, 5 in 2024-06-18T13:01:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:06 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:01:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:06 smithi167 ceph-mon[37701]: osdmap e32: 5 total, 5 up, 5 in 2024-06-18T13:01:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:06 smithi167 ceph-mon[29496]: purged_snaps scrub starts 2024-06-18T13:01:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:06 smithi167 ceph-mon[29496]: purged_snaps scrub ok 2024-06-18T13:01:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:06 smithi167 ceph-mon[29496]: osd.4 [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696] boot 2024-06-18T13:01:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:06 smithi167 ceph-mon[29496]: osdmap e31: 5 total, 5 up, 5 in 2024-06-18T13:01:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:06 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:01:06.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:06 smithi167 ceph-mon[29496]: osdmap e32: 5 total, 5 up, 5 in 2024-06-18T13:01:07.681 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_3 2024-06-18T13:01:07.916 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:01:08.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[37701]: Detected new or changed devices on smithi179 2024-06-18T13:01:08.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:08.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:08.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:08.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[37701]: Adjusting osd_memory_target on smithi179 to 17088M 2024-06-18T13:01:08.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:08.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:08.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:08.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:08.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[37701]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:08.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[37701]: osdmap e33: 5 total, 5 up, 5 in 2024-06-18T13:01:08.122 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[29496]: Detected new or changed devices on smithi179 2024-06-18T13:01:08.122 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:08.122 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:08.122 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:08.122 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[29496]: Adjusting osd_memory_target on smithi179 to 17088M 2024-06-18T13:01:08.122 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:08.122 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:08.122 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:08.122 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:08.123 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[29496]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:08.123 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:08 smithi167 ceph-mon[29496]: osdmap e33: 5 total, 5 up, 5 in 2024-06-18T13:01:08.356 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:08 smithi179 ceph-mon[33312]: Detected new or changed devices on smithi179 2024-06-18T13:01:08.356 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:08.356 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:08.356 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:08.356 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:08 smithi179 ceph-mon[33312]: Adjusting osd_memory_target on smithi179 to 17088M 2024-06-18T13:01:08.357 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:08.357 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:08.357 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:08.357 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:08.357 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:08 smithi179 ceph-mon[33312]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:08.357 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:08 smithi179 ceph-mon[33312]: osdmap e33: 5 total, 5 up, 5 in 2024-06-18T13:01:09.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.412+0000 7ff772117640 1 -- 172.21.15.179:0/1927726609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c101e80 msgr2=0x7ff76c108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:09.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.412+0000 7ff772117640 1 --2- 172.21.15.179:0/1927726609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c101e80 0x7ff76c108710 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff764009900 tx=0x7ff76402a410 comp rx=0 tx=0).stop 2024-06-18T13:01:09.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.412+0000 7ff772117640 1 -- 172.21.15.179:0/1927726609 shutdown_connections 2024-06-18T13:01:09.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.412+0000 7ff772117640 1 --2- 172.21.15.179:0/1927726609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c101e80 0x7ff76c108710 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:09.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.412+0000 7ff772117640 1 --2- 172.21.15.179:0/1927726609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff76c1014c0 0x7ff76c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:09.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.412+0000 7ff772117640 1 --2- 172.21.15.179:0/1927726609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff76c1002c0 0x7ff76c1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:09.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.412+0000 7ff772117640 1 -- 172.21.15.179:0/1927726609 >> 172.21.15.179:0/1927726609 conn(0x7ff76c0fba50 msgr2=0x7ff76c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:01:09.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.412+0000 7ff772117640 1 -- 172.21.15.179:0/1927726609 shutdown_connections 2024-06-18T13:01:09.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.412+0000 7ff772117640 1 -- 172.21.15.179:0/1927726609 wait complete. 2024-06-18T13:01:09.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.412+0000 7ff772117640 1 Processor -- start 2024-06-18T13:01:09.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.412+0000 7ff772117640 1 -- start start 2024-06-18T13:01:09.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff772117640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff76c1002c0 0x7ff76c104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:01:09.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff772117640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c1014c0 0x7ff76c1028a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:01:09.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff772117640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff76c101e80 0x7ff76c102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:01:09.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff772117640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c10b430 con 0x7ff76c101e80 2024-06-18T13:01:09.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff772117640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c10b2b0 con 0x7ff76c1014c0 2024-06-18T13:01:09.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff772117640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff76c10b5b0 con 0x7ff76c1002c0 2024-06-18T13:01:09.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff76affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c1014c0 0x7ff76c1028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:01:09.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff76affd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c1014c0 0x7ff76c1028a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44232/0 (socket says 172.21.15.179:44232) 2024-06-18T13:01:09.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff76affd640 1 -- 172.21.15.179:0/4265274460 learned_addr learned my addr 172.21.15.179:0/4265274460 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T13:01:09.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff76b7fe640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff76c1002c0 0x7ff76c104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:01:09.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff76bfff640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff76c101e80 0x7ff76c102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:01:09.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff76affd640 1 -- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff76c1002c0 msgr2=0x7ff76c104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:09.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff76affd640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff76c1002c0 0x7ff76c104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:09.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff76affd640 1 -- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff76c101e80 msgr2=0x7ff76c102de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:09.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff76affd640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff76c101e80 0x7ff76c102de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:09.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff76affd640 1 -- 172.21.15.179:0/4265274460 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76c1036a0 con 0x7ff76c1014c0 2024-06-18T13:01:09.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff76bfff640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff76c101e80 0x7ff76c102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T13:01:09.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff76b7fe640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff76c1002c0 0x7ff76c104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T13:01:09.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff76affd640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c1014c0 0x7ff76c1028a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff76000cce0 tx=0x7ff76000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:01:09.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff768ff9640 1 -- 172.21.15.179:0/4265274460 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760004b20 con 0x7ff76c1014c0 2024-06-18T13:01:09.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff772117640 1 -- 172.21.15.179:0/4265274460 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76c1a0da0 con 0x7ff76c1014c0 2024-06-18T13:01:09.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.413+0000 7ff768ff9640 1 -- 172.21.15.179:0/4265274460 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7ff760004cc0 con 0x7ff76c1014c0 2024-06-18T13:01:09.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.414+0000 7ff768ff9640 1 -- 172.21.15.179:0/4265274460 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76000f6b0 con 0x7ff76c1014c0 2024-06-18T13:01:09.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.414+0000 7ff772117640 1 -- 172.21.15.179:0/4265274460 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff76c1a1200 con 0x7ff76c1014c0 2024-06-18T13:01:09.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.415+0000 7ff772117640 1 -- 172.21.15.179:0/4265274460 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff76c1006c0 con 0x7ff76c1014c0 2024-06-18T13:01:09.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.415+0000 7ff768ff9640 1 -- 172.21.15.179:0/4265274460 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) ==== 92117+0+0 (secure 0 0 0) 0x7ff760005020 con 0x7ff76c1014c0 2024-06-18T13:01:09.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.416+0000 7ff768ff9640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff75406df50 0x7ff754070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:01:09.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.416+0000 7ff768ff9640 1 -- 172.21.15.179:0/4265274460 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 3991+0+0 (secure 0 0 0) 0x7ff76008d8c0 con 0x7ff76c1014c0 2024-06-18T13:01:09.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.416+0000 7ff76b7fe640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff75406df50 0x7ff754070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:01:09.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.416+0000 7ff76b7fe640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff75406df50 0x7ff754070410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff76c101320 tx=0x7ff75c00a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:01:09.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.420+0000 7ff768ff9640 1 -- 172.21.15.179:0/4265274460 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7ff760014030 con 0x7ff76c1014c0 2024-06-18T13:01:09.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:09.608+0000 7ff772117640 1 -- 172.21.15.179:0/4265274460 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff76c061de0 con 0x7ff75406df50 2024-06-18T13:01:09.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:09 smithi179 ceph-mon[33312]: osdmap e34: 5 total, 5 up, 5 in 2024-06-18T13:01:09.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:09 smithi179 ceph-mon[33312]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:09.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:09 smithi167 ceph-mon[29496]: osdmap e34: 5 total, 5 up, 5 in 2024-06-18T13:01:09.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:09 smithi167 ceph-mon[29496]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:09 smithi167 ceph-mon[37701]: osdmap e34: 5 total, 5 up, 5 in 2024-06-18T13:01:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:09 smithi167 ceph-mon[37701]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:10 smithi179 ceph-mon[33312]: osdmap e35: 5 total, 5 up, 5 in 2024-06-18T13:01:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:10 smithi179 ceph-mon[33312]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:01:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:10 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:01:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:10 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:01:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:10 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:10.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:10 smithi167 ceph-mon[29496]: osdmap e35: 5 total, 5 up, 5 in 2024-06-18T13:01:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:10 smithi167 ceph-mon[29496]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:01:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:10 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:01:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:10 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:01:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:10 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:10 smithi167 ceph-mon[37701]: osdmap e35: 5 total, 5 up, 5 in 2024-06-18T13:01:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:10 smithi167 ceph-mon[37701]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:01:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:10 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:01:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:10 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:01:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:10 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:11.605 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:11 smithi179 ceph-mon[33312]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:11.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:11 smithi167 ceph-mon[29496]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:11 smithi167 ceph-mon[37701]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:12.878 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.179:0/368051216' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45d6f34f-ce3b-4465-baa2-b49aec949910"}]: dispatch 2024-06-18T13:01:12.878 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45d6f34f-ce3b-4465-baa2-b49aec949910"}]: dispatch 2024-06-18T13:01:12.878 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45d6f34f-ce3b-4465-baa2-b49aec949910"}]': finished 2024-06-18T13:01:12.878 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:12 smithi167 ceph-mon[29496]: osdmap e36: 6 total, 5 up, 6 in 2024-06-18T13:01:12.878 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:12 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:12.878 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.179:0/3155194607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:01:12.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.179:0/368051216' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45d6f34f-ce3b-4465-baa2-b49aec949910"}]: dispatch 2024-06-18T13:01:12.879 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45d6f34f-ce3b-4465-baa2-b49aec949910"}]: dispatch 2024-06-18T13:01:12.879 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45d6f34f-ce3b-4465-baa2-b49aec949910"}]': finished 2024-06-18T13:01:12.879 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:12 smithi167 ceph-mon[37701]: osdmap e36: 6 total, 5 up, 6 in 2024-06-18T13:01:12.879 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:12 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:12.879 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.179:0/3155194607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:01:12.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.179:0/368051216' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45d6f34f-ce3b-4465-baa2-b49aec949910"}]: dispatch 2024-06-18T13:01:12.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45d6f34f-ce3b-4465-baa2-b49aec949910"}]: dispatch 2024-06-18T13:01:12.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45d6f34f-ce3b-4465-baa2-b49aec949910"}]': finished 2024-06-18T13:01:12.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:12 smithi179 ceph-mon[33312]: osdmap e36: 6 total, 5 up, 6 in 2024-06-18T13:01:12.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:12 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:12.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.179:0/3155194607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:01:13.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:13 smithi179 ceph-mon[33312]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-18T13:01:13.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:13 smithi167 ceph-mon[29496]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-18T13:01:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:13 smithi167 ceph-mon[37701]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-18T13:01:15.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:15 smithi179 ceph-mon[33312]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-18T13:01:15.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:15 smithi167 ceph-mon[29496]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-18T13:01:15.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:15 smithi167 ceph-mon[37701]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-18T13:01:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:17 smithi179 ceph-mon[33312]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-18T13:01:17.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:17 smithi167 ceph-mon[29496]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-18T13:01:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:17 smithi167 ceph-mon[37701]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-18T13:01:18.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:18 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-18T13:01:18.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:18 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:18 smithi179 ceph-mon[33312]: Deploying daemon osd.5 on smithi179 2024-06-18T13:01:18.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:18 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-18T13:01:18.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:18 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:18 smithi167 ceph-mon[29496]: Deploying daemon osd.5 on smithi179 2024-06-18T13:01:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:18 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-18T13:01:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:18 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:18 smithi167 ceph-mon[37701]: Deploying daemon osd.5 on smithi179 2024-06-18T13:01:19.776 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:19 smithi179 ceph-mon[33312]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:01:19.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:19 smithi167 ceph-mon[29496]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:01:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:19 smithi167 ceph-mon[37701]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:01:21.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:21 smithi179 ceph-mon[33312]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:01:21.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:21 smithi167 ceph-mon[29496]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:01:21.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:21 smithi167 ceph-mon[37701]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:01:23.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:22 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:23.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:22 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:23.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:22 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:23.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:22 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:23.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:22 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:23.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:22 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:23.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:22 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:23.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:22 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:23.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:22 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:23.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:23 smithi167 ceph-mon[29496]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:23 smithi167 ceph-mon[37701]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:24.132 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:23 smithi179 ceph-mon[33312]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 5 on host 'smithi179' 2024-06-18T13:01:26.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:26.119+0000 7ff768ff9640 1 -- 172.21.15.179:0/4265274460 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff76c061de0 con 0x7ff75406df50 2024-06-18T13:01:26.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:26.121+0000 7ff772117640 1 -- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff75406df50 msgr2=0x7ff754070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:26.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:26.121+0000 7ff772117640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff75406df50 0x7ff754070410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff76c101320 tx=0x7ff75c00a560 comp rx=0 tx=0).stop 2024-06-18T13:01:26.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:26.121+0000 7ff772117640 1 -- 172.21.15.179:0/4265274460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c1014c0 msgr2=0x7ff76c1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:26.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:26.121+0000 7ff772117640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c1014c0 0x7ff76c1028a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff76000cce0 tx=0x7ff76000a620 comp rx=0 tx=0).stop 2024-06-18T13:01:26.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:26.121+0000 7ff772117640 1 -- 172.21.15.179:0/4265274460 shutdown_connections 2024-06-18T13:01:26.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:26.121+0000 7ff772117640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ff75406df50 0x7ff754070410 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:26.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:26.121+0000 7ff772117640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff76c101e80 0x7ff76c102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:26.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:26.121+0000 7ff772117640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c1014c0 0x7ff76c1028a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:26.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:26.121+0000 7ff772117640 1 --2- 172.21.15.179:0/4265274460 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff76c1002c0 0x7ff76c104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:26.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:26.121+0000 7ff772117640 1 -- 172.21.15.179:0/4265274460 >> 172.21.15.179:0/4265274460 conn(0x7ff76c0fba50 msgr2=0x7ff76c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:01:26.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:26.122+0000 7ff772117640 1 -- 172.21.15.179:0/4265274460 shutdown_connections 2024-06-18T13:01:26.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:26.122+0000 7ff772117640 1 -- 172.21.15.179:0/4265274460 wait complete. 2024-06-18T13:01:26.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:26.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:26.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:26 smithi179 ceph-mon[33312]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:26.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:26 smithi179 ceph-mon[33312]: from='osd.5 [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-18T13:01:26.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:26 smithi179 ceph-mon[33312]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-18T13:01:26.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:26.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:26 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:26.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:26.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[29496]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:26.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[29496]: from='osd.5 [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-18T13:01:26.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[29496]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-18T13:01:26.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:26.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:26.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:26.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[37701]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:26.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[37701]: from='osd.5 [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-18T13:01:26.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[37701]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-18T13:01:26.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:26.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:26 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:26.777 DEBUG:teuthology.orchestra.run.smithi179:osd.5> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.5.service 2024-06-18T13:01:26.779 INFO:tasks.cephadm:Deploying osd.6 on smithi179 with /dev/vg_nvme/lv_2... 2024-06-18T13:01:26.780 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-18T13:01:27.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:27 smithi179 ceph-mon[33312]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-18T13:01:27.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:27 smithi179 ceph-mon[33312]: osdmap e37: 6 total, 5 up, 6 in 2024-06-18T13:01:27.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:27 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:27.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:27 smithi179 ceph-mon[33312]: from='osd.5 [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:27.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:27 smithi179 ceph-mon[33312]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:27.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:27 smithi167 ceph-mon[29496]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-18T13:01:27.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:27 smithi167 ceph-mon[29496]: osdmap e37: 6 total, 5 up, 6 in 2024-06-18T13:01:27.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:27 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:27.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:27 smithi167 ceph-mon[29496]: from='osd.5 [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:27.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:27 smithi167 ceph-mon[29496]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:27.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:27 smithi167 ceph-mon[37701]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-18T13:01:27.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:27 smithi167 ceph-mon[37701]: osdmap e37: 6 total, 5 up, 6 in 2024-06-18T13:01:27.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:27 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:27.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:27 smithi167 ceph-mon[37701]: from='osd.5 [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:27.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:27 smithi167 ceph-mon[37701]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:27.492 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:01:27.886 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:01:27 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:01:27.615+0000 7f61197af640 -1 osd.5 0 waiting for initial osdmap 2024-06-18T13:01:27.887 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:01:27 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:01:27.619+0000 7f6110dbb640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-18T13:01:28.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:28 smithi167 ceph-mon[37701]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-18T13:01:28.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:28 smithi167 ceph-mon[37701]: osdmap e38: 6 total, 5 up, 6 in 2024-06-18T13:01:28.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:28 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:28.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:28 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:28.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:28 smithi167 ceph-mon[37701]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:28.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:28 smithi167 ceph-mon[37701]: from='osd.5 ' entity='osd.5' 2024-06-18T13:01:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:28 smithi167 ceph-mon[29496]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-18T13:01:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:28 smithi167 ceph-mon[29496]: osdmap e38: 6 total, 5 up, 6 in 2024-06-18T13:01:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:28 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:28 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:28 smithi167 ceph-mon[29496]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:28 smithi167 ceph-mon[29496]: from='osd.5 ' entity='osd.5' 2024-06-18T13:01:28.532 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:28 smithi179 ceph-mon[33312]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-18T13:01:28.532 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:28 smithi179 ceph-mon[33312]: osdmap e38: 6 total, 5 up, 6 in 2024-06-18T13:01:28.532 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:28 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:28.532 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:28 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:28.532 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:28 smithi179 ceph-mon[33312]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-18T13:01:28.532 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:28 smithi179 ceph-mon[33312]: from='osd.5 ' entity='osd.5' 2024-06-18T13:01:29.466 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:29 smithi179 ceph-mon[33312]: purged_snaps scrub starts 2024-06-18T13:01:29.466 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:29 smithi179 ceph-mon[33312]: purged_snaps scrub ok 2024-06-18T13:01:29.466 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:29 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:29.466 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:29 smithi179 ceph-mon[33312]: osd.5 [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992] boot 2024-06-18T13:01:29.466 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:29 smithi179 ceph-mon[33312]: osdmap e39: 6 total, 6 up, 6 in 2024-06-18T13:01:29.466 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:29 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:29.466 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:29 smithi179 ceph-mon[33312]: osdmap e40: 6 total, 6 up, 6 in 2024-06-18T13:01:29.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:29 smithi167 ceph-mon[29496]: purged_snaps scrub starts 2024-06-18T13:01:29.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:29 smithi167 ceph-mon[29496]: purged_snaps scrub ok 2024-06-18T13:01:29.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:29 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:29.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:29 smithi167 ceph-mon[29496]: osd.5 [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992] boot 2024-06-18T13:01:29.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:29 smithi167 ceph-mon[29496]: osdmap e39: 6 total, 6 up, 6 in 2024-06-18T13:01:29.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:29 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:29.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:29 smithi167 ceph-mon[29496]: osdmap e40: 6 total, 6 up, 6 in 2024-06-18T13:01:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:29 smithi167 ceph-mon[37701]: purged_snaps scrub starts 2024-06-18T13:01:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:29 smithi167 ceph-mon[37701]: purged_snaps scrub ok 2024-06-18T13:01:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:29 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:29 smithi167 ceph-mon[37701]: osd.5 [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992] boot 2024-06-18T13:01:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:29 smithi167 ceph-mon[37701]: osdmap e39: 6 total, 6 up, 6 in 2024-06-18T13:01:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:29 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:01:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:29 smithi167 ceph-mon[37701]: osdmap e40: 6 total, 6 up, 6 in 2024-06-18T13:01:29.782 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-18T13:01:29.876 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-18T13:01:29.891 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-06-18T13:01:29.891 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-06-18T13:01:29.891 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130692 s, 802 MB/s 2024-06-18T13:01:29.891 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-06-18T13:01:30.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:30 smithi167 ceph-mon[29496]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:30.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:30 smithi167 ceph-mon[29496]: osdmap e41: 6 total, 6 up, 6 in 2024-06-18T13:01:30.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:30 smithi167 ceph-mon[37701]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:30.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:30 smithi167 ceph-mon[37701]: osdmap e41: 6 total, 6 up, 6 in 2024-06-18T13:01:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:30 smithi179 ceph-mon[33312]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:30 smithi179 ceph-mon[33312]: osdmap e41: 6 total, 6 up, 6 in 2024-06-18T13:01:30.600 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_2 2024-06-18T13:01:31.178 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:01:31.300 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:31 smithi179 ceph-mon[33312]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-18T13:01:31.300 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:31 smithi179 ceph-mon[33312]: osdmap e42: 6 total, 6 up, 6 in 2024-06-18T13:01:31.300 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:31 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:31.300 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:31 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:31.300 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:31 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:31.301 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:31 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:31.301 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:31 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:31.301 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:31 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:31.301 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:31 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:31.301 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:31 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:31.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[29496]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-18T13:01:31.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[29496]: osdmap e42: 6 total, 6 up, 6 in 2024-06-18T13:01:31.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:31.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:31.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:31.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:31.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:31.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:31.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:31.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:31.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[37701]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-18T13:01:31.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[37701]: osdmap e42: 6 total, 6 up, 6 in 2024-06-18T13:01:31.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:31.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:31.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:31.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:31.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:31.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:31.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:31.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:31 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:32.367 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:32 smithi179 ceph-mon[33312]: Detected new or changed devices on smithi179 2024-06-18T13:01:32.368 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:32 smithi179 ceph-mon[33312]: Adjusting osd_memory_target on smithi179 to 8544M 2024-06-18T13:01:32.368 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:32 smithi179 ceph-mon[33312]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:32.368 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:32 smithi179 ceph-mon[33312]: osdmap e43: 6 total, 6 up, 6 in 2024-06-18T13:01:32.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:32 smithi167 ceph-mon[29496]: Detected new or changed devices on smithi179 2024-06-18T13:01:32.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:32 smithi167 ceph-mon[29496]: Adjusting osd_memory_target on smithi179 to 8544M 2024-06-18T13:01:32.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:32 smithi167 ceph-mon[29496]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:32.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:32 smithi167 ceph-mon[29496]: osdmap e43: 6 total, 6 up, 6 in 2024-06-18T13:01:32.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:32 smithi167 ceph-mon[37701]: Detected new or changed devices on smithi179 2024-06-18T13:01:32.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:32 smithi167 ceph-mon[37701]: Adjusting osd_memory_target on smithi179 to 8544M 2024-06-18T13:01:32.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:32 smithi167 ceph-mon[37701]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:32.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:32 smithi167 ceph-mon[37701]: osdmap e43: 6 total, 6 up, 6 in 2024-06-18T13:01:32.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 -- 172.21.15.179:0/868667581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83b4101e10 msgr2=0x7f83b4104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:32.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 --2- 172.21.15.179:0/868667581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83b4101e10 0x7f83b4104200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009900 tx=0x7f83b002a410 comp rx=0 tx=0).stop 2024-06-18T13:01:32.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 -- 172.21.15.179:0/868667581 shutdown_connections 2024-06-18T13:01:32.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 --2- 172.21.15.179:0/868667581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83b4107070 0x7f83b41094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:32.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 --2- 172.21.15.179:0/868667581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83b4104740 0x7f83b4106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:32.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 --2- 172.21.15.179:0/868667581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83b4101e10 0x7f83b4104200 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:32.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 -- 172.21.15.179:0/868667581 >> 172.21.15.179:0/868667581 conn(0x7f83b40fba30 msgr2=0x7f83b40fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:01:32.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 -- 172.21.15.179:0/868667581 shutdown_connections 2024-06-18T13:01:32.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 -- 172.21.15.179:0/868667581 wait complete. 2024-06-18T13:01:32.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 Processor -- start 2024-06-18T13:01:32.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 -- start start 2024-06-18T13:01:32.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83b4101e10 0x7f83b4101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:01:32.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83b4104740 0x7f83b40ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:01:32.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83b4107070 0x7f83b4100300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:01:32.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f83b410c4f0 con 0x7f83b4101e10 2024-06-18T13:01:32.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f83b410c370 con 0x7f83b4104740 2024-06-18T13:01:32.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.713+0000 7f83bc40d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f83b410c670 con 0x7f83b4107070 2024-06-18T13:01:32.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83b99a8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83b4104740 0x7f83b40ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:01:32.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83b99a8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83b4104740 0x7f83b40ffdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52418/0 (socket says 172.21.15.179:52418) 2024-06-18T13:01:32.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83b99a8640 1 -- 172.21.15.179:0/4054992855 learned_addr learned my addr 172.21.15.179:0/4054992855 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T13:01:32.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83ba1a9640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83b4101e10 0x7f83b4101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:01:32.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83ba9aa640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83b4107070 0x7f83b4100300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:01:32.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83b99a8640 1 -- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83b4107070 msgr2=0x7f83b4100300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:32.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83b99a8640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83b4107070 0x7f83b4100300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:32.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83b99a8640 1 -- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83b4101e10 msgr2=0x7f83b4101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:32.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83b99a8640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83b4101e10 0x7f83b4101770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:32.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83b99a8640 1 -- 172.21.15.179:0/4054992855 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b4100bc0 con 0x7f83b4104740 2024-06-18T13:01:32.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83ba9aa640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83b4107070 0x7f83b4100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T13:01:32.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83b99a8640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83b4104740 0x7f83b40ffdc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f83a400c970 tx=0x7f83a400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:01:32.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83ab7fe640 1 -- 172.21.15.179:0/4054992855 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4007bb0 con 0x7f83b4104740 2024-06-18T13:01:32.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83ab7fe640 1 -- 172.21.15.179:0/4054992855 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f83a4007d50 con 0x7f83b4104740 2024-06-18T13:01:32.839 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83bc40d640 1 -- 172.21.15.179:0/4054992855 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b41a0d90 con 0x7f83b4104740 2024-06-18T13:01:32.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83ab7fe640 1 -- 172.21.15.179:0/4054992855 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4018660 con 0x7f83b4104740 2024-06-18T13:01:32.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.714+0000 7f83bc40d640 1 -- 172.21.15.179:0/4054992855 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b41a11f0 con 0x7f83b4104740 2024-06-18T13:01:32.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.715+0000 7f83bc40d640 1 -- 172.21.15.179:0/4054992855 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b4104b20 con 0x7f83b4104740 2024-06-18T13:01:32.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.716+0000 7f83ab7fe640 1 -- 172.21.15.179:0/4054992855 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) ==== 92117+0+0 (secure 0 0 0) 0x7f83a400f450 con 0x7f83b4104740 2024-06-18T13:01:32.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.716+0000 7f83ab7fe640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f83a006df50 0x7f83a0070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:01:32.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.716+0000 7f83ab7fe640 1 -- 172.21.15.179:0/4054992855 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4423+0+0 (secure 0 0 0) 0x7f83a408d920 con 0x7f83b4104740 2024-06-18T13:01:32.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.716+0000 7f83ba1a9640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f83a006df50 0x7f83a0070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:01:32.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.717+0000 7f83ba1a9640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f83a006df50 0x7f83a0070410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f83b0000c00 tx=0x7f83b00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:01:32.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.721+0000 7f83ab7fe640 1 -- 172.21.15.179:0/4054992855 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f83a4014030 con 0x7f83b4104740 2024-06-18T13:01:32.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:32.907+0000 7f83bc40d640 1 -- 172.21.15.179:0/4054992855 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f83b4061de0 con 0x7f83a006df50 2024-06-18T13:01:33.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:33 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:01:33.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:33 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:01:33.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:33 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:33.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:33 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:01:33.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:33 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:01:33.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:33 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:33.507 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:33 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:01:33.507 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:33 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:01:33.507 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:33 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:34.317 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:34 smithi179 ceph-mon[33312]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:01:34.318 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:34 smithi179 ceph-mon[33312]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:34.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:34 smithi167 ceph-mon[29496]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:01:34.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:34 smithi167 ceph-mon[29496]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:34.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:34 smithi167 ceph-mon[37701]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:01:34.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:34 smithi167 ceph-mon[37701]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:35.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.179:0/547137419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06a47b6b-eac6-440b-9037-b55bfaf29e5c"}]: dispatch 2024-06-18T13:01:35.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06a47b6b-eac6-440b-9037-b55bfaf29e5c"}]: dispatch 2024-06-18T13:01:35.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06a47b6b-eac6-440b-9037-b55bfaf29e5c"}]': finished 2024-06-18T13:01:35.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:35 smithi167 ceph-mon[29496]: osdmap e44: 7 total, 6 up, 7 in 2024-06-18T13:01:35.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:35 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:35.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.179:0/547137419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06a47b6b-eac6-440b-9037-b55bfaf29e5c"}]: dispatch 2024-06-18T13:01:35.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06a47b6b-eac6-440b-9037-b55bfaf29e5c"}]: dispatch 2024-06-18T13:01:35.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06a47b6b-eac6-440b-9037-b55bfaf29e5c"}]': finished 2024-06-18T13:01:35.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:35 smithi167 ceph-mon[37701]: osdmap e44: 7 total, 6 up, 7 in 2024-06-18T13:01:35.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:35 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:35.531 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.179:0/547137419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06a47b6b-eac6-440b-9037-b55bfaf29e5c"}]: dispatch 2024-06-18T13:01:35.532 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06a47b6b-eac6-440b-9037-b55bfaf29e5c"}]: dispatch 2024-06-18T13:01:35.532 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06a47b6b-eac6-440b-9037-b55bfaf29e5c"}]': finished 2024-06-18T13:01:35.532 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:35 smithi179 ceph-mon[33312]: osdmap e44: 7 total, 6 up, 7 in 2024-06-18T13:01:35.532 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:35 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:36.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:36 smithi179 ceph-mon[33312]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-18T13:01:36.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.179:0/2249301410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:01:36.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:36 smithi167 ceph-mon[29496]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-18T13:01:36.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.179:0/2249301410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:01:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:36 smithi167 ceph-mon[37701]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-18T13:01:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.179:0/2249301410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:01:37.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:37 smithi179 ceph-mon[33312]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-18T13:01:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:37 smithi179 ceph-mon[33312]: Cluster is now healthy 2024-06-18T13:01:37.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:37 smithi167 ceph-mon[29496]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-18T13:01:37.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:37 smithi167 ceph-mon[29496]: Cluster is now healthy 2024-06-18T13:01:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:37 smithi167 ceph-mon[37701]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-18T13:01:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:37 smithi167 ceph-mon[37701]: Cluster is now healthy 2024-06-18T13:01:38.580 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:38 smithi179 ceph-mon[33312]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-18T13:01:38.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:38 smithi167 ceph-mon[29496]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-18T13:01:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:38 smithi167 ceph-mon[37701]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-18T13:01:39.779 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:39 smithi179 ceph-mon[33312]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-18T13:01:39.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:39 smithi167 ceph-mon[29496]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-18T13:01:39.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:39 smithi167 ceph-mon[37701]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-18T13:01:41.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:41 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-18T13:01:41.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:41 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:41.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:41 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-18T13:01:41.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:41 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:41.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:41 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-18T13:01:41.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:41 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:42.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:42 smithi179 ceph-mon[33312]: Deploying daemon osd.6 on smithi179 2024-06-18T13:01:42.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:42 smithi179 ceph-mon[33312]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:01:42.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:42 smithi167 ceph-mon[29496]: Deploying daemon osd.6 on smithi179 2024-06-18T13:01:42.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:42 smithi167 ceph-mon[29496]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:01:42.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:42 smithi167 ceph-mon[37701]: Deploying daemon osd.6 on smithi179 2024-06-18T13:01:42.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:42 smithi167 ceph-mon[37701]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:01:43.800 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:43 smithi179 ceph-mon[33312]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:01:43.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:43 smithi167 ceph-mon[29496]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:01:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:43 smithi167 ceph-mon[37701]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:01:45.712 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:45 smithi179 ceph-mon[33312]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:45.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:45 smithi167 ceph-mon[37701]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:45.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:45 smithi167 ceph-mon[29496]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:46.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:46 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:46.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:46 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:46.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:46 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:46.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:46 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:46.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:46 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:46 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:46 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:46 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:46 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:47.712 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:47 smithi179 ceph-mon[33312]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:47.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:47 smithi167 ceph-mon[29496]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:47.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:47 smithi167 ceph-mon[37701]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:48.922 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 6 on host 'smithi179' 2024-06-18T13:01:48.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:48.922+0000 7f83ab7fe640 1 -- 172.21.15.179:0/4054992855 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f83b4061de0 con 0x7f83a006df50 2024-06-18T13:01:48.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:48.924+0000 7f83bc40d640 1 -- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f83a006df50 msgr2=0x7f83a0070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:48.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:48.924+0000 7f83bc40d640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f83a006df50 0x7f83a0070410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f83b0000c00 tx=0x7f83b00058d0 comp rx=0 tx=0).stop 2024-06-18T13:01:48.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:48.924+0000 7f83bc40d640 1 -- 172.21.15.179:0/4054992855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83b4104740 msgr2=0x7f83b40ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:48.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:48.924+0000 7f83bc40d640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83b4104740 0x7f83b40ffdc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f83a400c970 tx=0x7f83a400ce30 comp rx=0 tx=0).stop 2024-06-18T13:01:48.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:48.924+0000 7f83bc40d640 1 -- 172.21.15.179:0/4054992855 shutdown_connections 2024-06-18T13:01:48.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:48.924+0000 7f83bc40d640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f83a006df50 0x7f83a0070410 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:48.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:48.924+0000 7f83bc40d640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83b4107070 0x7f83b4100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:48.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:48.924+0000 7f83bc40d640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83b4104740 0x7f83b40ffdc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:48.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:48.924+0000 7f83bc40d640 1 --2- 172.21.15.179:0/4054992855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83b4101e10 0x7f83b4101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:48.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:48.924+0000 7f83bc40d640 1 -- 172.21.15.179:0/4054992855 >> 172.21.15.179:0/4054992855 conn(0x7f83b40fba30 msgr2=0x7f83b4102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:01:48.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:48.924+0000 7f83bc40d640 1 -- 172.21.15.179:0/4054992855 shutdown_connections 2024-06-18T13:01:48.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:48.924+0000 7f83bc40d640 1 -- 172.21.15.179:0/4054992855 wait complete. 2024-06-18T13:01:49.486 DEBUG:teuthology.orchestra.run.smithi179:osd.6> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.6.service 2024-06-18T13:01:49.488 INFO:tasks.cephadm:Deploying osd.7 on smithi179 with /dev/vg_nvme/lv_1... 2024-06-18T13:01:49.488 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-18T13:01:49.786 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:01:50.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:49 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:50.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:49 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:50.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:49 smithi167 ceph-mon[37701]: from='osd.6 [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-18T13:01:50.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:49 smithi167 ceph-mon[37701]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-18T13:01:50.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:49 smithi167 ceph-mon[37701]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:50.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:49 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:50.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:49 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:49 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:49 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:49 smithi167 ceph-mon[29496]: from='osd.6 [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-18T13:01:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:49 smithi167 ceph-mon[29496]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-18T13:01:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:49 smithi167 ceph-mon[29496]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:49 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:50.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:49 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:50.268 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:49 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:50.268 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:49 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:50.268 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:49 smithi179 ceph-mon[33312]: from='osd.6 [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-18T13:01:50.268 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:49 smithi179 ceph-mon[33312]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-18T13:01:50.268 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:49 smithi179 ceph-mon[33312]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:50.269 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:49 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:50.269 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:49 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:51.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:50 smithi179 ceph-mon[33312]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-18T13:01:51.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:50 smithi179 ceph-mon[33312]: osdmap e45: 7 total, 6 up, 7 in 2024-06-18T13:01:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:50 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:50 smithi179 ceph-mon[33312]: from='osd.6 [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:50 smithi179 ceph-mon[33312]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:51.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:50 smithi167 ceph-mon[29496]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-18T13:01:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:50 smithi167 ceph-mon[29496]: osdmap e45: 7 total, 6 up, 7 in 2024-06-18T13:01:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:50 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:50 smithi167 ceph-mon[29496]: from='osd.6 [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:50 smithi167 ceph-mon[29496]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:51.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:50 smithi167 ceph-mon[37701]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-18T13:01:51.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:50 smithi167 ceph-mon[37701]: osdmap e45: 7 total, 6 up, 7 in 2024-06-18T13:01:51.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:50 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:51.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:50 smithi167 ceph-mon[37701]: from='osd.6 [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:51.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:50 smithi167 ceph-mon[37701]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:01:51.445 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:01:51 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:01:51.373+0000 7f272f438640 -1 osd.6 0 waiting for initial osdmap 2024-06-18T13:01:51.445 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:01:51 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:01:51.379+0000 7f2727258640 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-18T13:01:51.758 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-18T13:01:51.842 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-18T13:01:51.857 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-06-18T13:01:51.857 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-06-18T13:01:51.857 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122174 s, 858 MB/s 2024-06-18T13:01:51.857 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-06-18T13:01:52.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:51 smithi179 ceph-mon[33312]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-18T13:01:52.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:51 smithi179 ceph-mon[33312]: osdmap e46: 7 total, 6 up, 7 in 2024-06-18T13:01:52.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:51 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:51 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:51 smithi179 ceph-mon[33312]: from='osd.6 ' entity='osd.6' 2024-06-18T13:01:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:51 smithi179 ceph-mon[33312]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:51 smithi179 ceph-mon[33312]: osd.6 [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065] boot 2024-06-18T13:01:52.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:51 smithi167 ceph-mon[29496]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-18T13:01:52.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:51 smithi167 ceph-mon[29496]: osdmap e46: 7 total, 6 up, 7 in 2024-06-18T13:01:52.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:52.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:52.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:51 smithi167 ceph-mon[29496]: from='osd.6 ' entity='osd.6' 2024-06-18T13:01:52.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:51 smithi167 ceph-mon[29496]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:52.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:51 smithi167 ceph-mon[29496]: osd.6 [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065] boot 2024-06-18T13:01:52.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:51 smithi167 ceph-mon[37701]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-18T13:01:52.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:51 smithi167 ceph-mon[37701]: osdmap e46: 7 total, 6 up, 7 in 2024-06-18T13:01:52.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:51 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:52.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:51 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:52.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:51 smithi167 ceph-mon[37701]: from='osd.6 ' entity='osd.6' 2024-06-18T13:01:52.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:51 smithi167 ceph-mon[37701]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-18T13:01:52.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:51 smithi167 ceph-mon[37701]: osd.6 [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065] boot 2024-06-18T13:01:52.497 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_1 2024-06-18T13:01:52.923 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:01:53.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:52 smithi179 ceph-mon[33312]: purged_snaps scrub starts 2024-06-18T13:01:53.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:52 smithi179 ceph-mon[33312]: purged_snaps scrub ok 2024-06-18T13:01:53.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:52 smithi179 ceph-mon[33312]: osdmap e47: 7 total, 7 up, 7 in 2024-06-18T13:01:53.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:52 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:53.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:52 smithi179 ceph-mon[33312]: osdmap e48: 7 total, 7 up, 7 in 2024-06-18T13:01:53.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:52 smithi167 ceph-mon[37701]: purged_snaps scrub starts 2024-06-18T13:01:53.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:52 smithi167 ceph-mon[37701]: purged_snaps scrub ok 2024-06-18T13:01:53.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:52 smithi167 ceph-mon[37701]: osdmap e47: 7 total, 7 up, 7 in 2024-06-18T13:01:53.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:52 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:53.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:52 smithi167 ceph-mon[37701]: osdmap e48: 7 total, 7 up, 7 in 2024-06-18T13:01:53.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:52 smithi167 ceph-mon[29496]: purged_snaps scrub starts 2024-06-18T13:01:53.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:52 smithi167 ceph-mon[29496]: purged_snaps scrub ok 2024-06-18T13:01:53.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:52 smithi167 ceph-mon[29496]: osdmap e47: 7 total, 7 up, 7 in 2024-06-18T13:01:53.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:52 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:01:53.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:52 smithi167 ceph-mon[29496]: osdmap e48: 7 total, 7 up, 7 in 2024-06-18T13:01:54.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:53 smithi167 ceph-mon[29496]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 625 GiB / 626 GiB avail 2024-06-18T13:01:54.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:53 smithi167 ceph-mon[37701]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 625 GiB / 626 GiB avail 2024-06-18T13:01:54.268 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:53 smithi179 ceph-mon[33312]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 625 GiB / 626 GiB avail 2024-06-18T13:01:55.016 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.014+0000 7ffb38efa640 1 -- 172.21.15.179:0/3395455433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb341002c0 msgr2=0x7ffb341006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:55.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.014+0000 7ffb38efa640 1 --2- 172.21.15.179:0/3395455433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb341002c0 0x7ffb341006c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009900 tx=0x7ffb2402a410 comp rx=0 tx=0).stop 2024-06-18T13:01:55.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.014+0000 7ffb38efa640 1 -- 172.21.15.179:0/3395455433 shutdown_connections 2024-06-18T13:01:55.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.014+0000 7ffb38efa640 1 --2- 172.21.15.179:0/3395455433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb34101e80 0x7ffb34108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:55.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.014+0000 7ffb38efa640 1 --2- 172.21.15.179:0/3395455433 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb341014c0 0x7ffb34101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:55.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.014+0000 7ffb38efa640 1 --2- 172.21.15.179:0/3395455433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb341002c0 0x7ffb341006c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:55.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.014+0000 7ffb38efa640 1 -- 172.21.15.179:0/3395455433 >> 172.21.15.179:0/3395455433 conn(0x7ffb340fba50 msgr2=0x7ffb340fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:01:55.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.014+0000 7ffb38efa640 1 -- 172.21.15.179:0/3395455433 shutdown_connections 2024-06-18T13:01:55.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.014+0000 7ffb38efa640 1 -- 172.21.15.179:0/3395455433 wait complete. 2024-06-18T13:01:55.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb38efa640 1 Processor -- start 2024-06-18T13:01:55.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb38efa640 1 -- start start 2024-06-18T13:01:55.017 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb38efa640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb341002c0 0x7ffb3406aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:01:55.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb38efa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb341014c0 0x7ffb3419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:01:55.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb38efa640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb34101e80 0x7ffb3419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:01:55.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb38efa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3410b4e0 con 0x7ffb341014c0 2024-06-18T13:01:55.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb38efa640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3410b360 con 0x7ffb34101e80 2024-06-18T13:01:55.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb38efa640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ffb3410b660 con 0x7ffb341002c0 2024-06-18T13:01:55.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb32d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb34101e80 0x7ffb3419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:01:55.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb32d9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb34101e80 0x7ffb3419e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:42794/0 (socket says 172.21.15.179:42794) 2024-06-18T13:01:55.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb32d9d640 1 -- 172.21.15.179:0/823725749 learned_addr learned my addr 172.21.15.179:0/823725749 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T13:01:55.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb31d9b640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb341014c0 0x7ffb3419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:01:55.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb3259c640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb341002c0 0x7ffb3406aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:01:55.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb32d9d640 1 -- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb341002c0 msgr2=0x7ffb3406aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:55.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb32d9d640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb341002c0 0x7ffb3406aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:55.018 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb32d9d640 1 -- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb341014c0 msgr2=0x7ffb3419bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:01:55.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb32d9d640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb341014c0 0x7ffb3419bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:01:55.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb32d9d640 1 -- 172.21.15.179:0/823725749 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb34195920 con 0x7ffb34101e80 2024-06-18T13:01:55.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb31d9b640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb341014c0 0x7ffb3419bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-18T13:01:55.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.015+0000 7ffb32d9d640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb34101e80 0x7ffb3419e170 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00b970 tx=0x7ffb2c00be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:01:55.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.016+0000 7ffb2b7fe640 1 -- 172.21.15.179:0/823725749 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c019070 con 0x7ffb34101e80 2024-06-18T13:01:55.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.016+0000 7ffb2b7fe640 1 -- 172.21.15.179:0/823725749 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7ffb2c014440 con 0x7ffb34101e80 2024-06-18T13:01:55.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.016+0000 7ffb38efa640 1 -- 172.21.15.179:0/823725749 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb34195bb0 con 0x7ffb34101e80 2024-06-18T13:01:55.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.016+0000 7ffb2b7fe640 1 -- 172.21.15.179:0/823725749 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0075d0 con 0x7ffb34101e80 2024-06-18T13:01:55.019 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.016+0000 7ffb38efa640 1 -- 172.21.15.179:0/823725749 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb34196120 con 0x7ffb34101e80 2024-06-18T13:01:55.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.017+0000 7ffb38efa640 1 -- 172.21.15.179:0/823725749 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb34100740 con 0x7ffb34101e80 2024-06-18T13:01:55.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.017+0000 7ffb2b7fe640 1 -- 172.21.15.179:0/823725749 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) ==== 92117+0+0 (secure 0 0 0) 0x7ffb2c01c070 con 0x7ffb34101e80 2024-06-18T13:01:55.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.018+0000 7ffb2b7fe640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ffb2006df50 0x7ffb20070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:01:55.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.018+0000 7ffb2b7fe640 1 -- 172.21.15.179:0/823725749 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4855+0+0 (secure 0 0 0) 0x7ffb2c08d330 con 0x7ffb34101e80 2024-06-18T13:01:55.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.018+0000 7ffb3259c640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ffb2006df50 0x7ffb20070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:01:55.020 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.018+0000 7ffb3259c640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ffb2006df50 0x7ffb20070410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb240096c0 tx=0x7ffb24005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:01:55.023 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.022+0000 7ffb2b7fe640 1 -- 172.21.15.179:0/823725749 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7ffb2c05b130 con 0x7ffb34101e80 2024-06-18T13:01:55.125 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:54 smithi179 ceph-mon[33312]: osdmap e49: 7 total, 7 up, 7 in 2024-06-18T13:01:55.214 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:01:55.213+0000 7ffb38efa640 1 -- 172.21.15.179:0/823725749 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ffb34061de0 con 0x7ffb2006df50 2024-06-18T13:01:55.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:54 smithi167 ceph-mon[29496]: osdmap e49: 7 total, 7 up, 7 in 2024-06-18T13:01:55.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:54 smithi167 ceph-mon[37701]: osdmap e49: 7 total, 7 up, 7 in 2024-06-18T13:01:56.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: osdmap e50: 7 total, 7 up, 7 in 2024-06-18T13:01:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:01:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:01:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:01:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: Detected new or changed devices on smithi179 2024-06-18T13:01:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: Adjusting osd_memory_target on smithi179 to 5696M 2024-06-18T13:01:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:56.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:56.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:56.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 385 MiB used, 625 GiB / 626 GiB avail 2024-06-18T13:01:56.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:56.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:56.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:56.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:55 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:56.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: osdmap e50: 7 total, 7 up, 7 in 2024-06-18T13:01:56.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:01:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:01:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:01:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: Detected new or changed devices on smithi179 2024-06-18T13:01:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: Adjusting osd_memory_target on smithi179 to 5696M 2024-06-18T13:01:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:56.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 385 MiB used, 625 GiB / 626 GiB avail 2024-06-18T13:01:56.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:56.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:56.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:56.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:56.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: osdmap e50: 7 total, 7 up, 7 in 2024-06-18T13:01:56.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:01:56.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-18T13:01:56.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-18T13:01:56.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:56.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: Detected new or changed devices on smithi179 2024-06-18T13:01:56.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:56.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:56.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:56.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:56.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:01:56.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: Adjusting osd_memory_target on smithi179 to 5696M 2024-06-18T13:01:56.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:56.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:56.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:56.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:56.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 385 MiB used, 625 GiB / 626 GiB avail 2024-06-18T13:01:56.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:01:56.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:01:56.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:01:56.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:55 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:01:57.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:56 smithi179 ceph-mon[33312]: osdmap e51: 7 total, 7 up, 7 in 2024-06-18T13:01:57.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:56 smithi167 ceph-mon[29496]: osdmap e51: 7 total, 7 up, 7 in 2024-06-18T13:01:57.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:56 smithi167 ceph-mon[37701]: osdmap e51: 7 total, 7 up, 7 in 2024-06-18T13:01:58.075 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.179:0/1430935446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717295cb-4786-41e4-9ac0-40fcea4b0375"}]: dispatch 2024-06-18T13:01:58.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717295cb-4786-41e4-9ac0-40fcea4b0375"}]: dispatch 2024-06-18T13:01:58.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "717295cb-4786-41e4-9ac0-40fcea4b0375"}]': finished 2024-06-18T13:01:58.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:57 smithi179 ceph-mon[33312]: osdmap e52: 8 total, 7 up, 8 in 2024-06-18T13:01:58.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:57 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:01:58.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:57 smithi179 ceph-mon[33312]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-18T13:01:58.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.179:0/1430935446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717295cb-4786-41e4-9ac0-40fcea4b0375"}]: dispatch 2024-06-18T13:01:58.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717295cb-4786-41e4-9ac0-40fcea4b0375"}]: dispatch 2024-06-18T13:01:58.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "717295cb-4786-41e4-9ac0-40fcea4b0375"}]': finished 2024-06-18T13:01:58.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:57 smithi167 ceph-mon[29496]: osdmap e52: 8 total, 7 up, 8 in 2024-06-18T13:01:58.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:57 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:01:58.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:57 smithi167 ceph-mon[29496]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-18T13:01:58.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.179:0/1430935446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717295cb-4786-41e4-9ac0-40fcea4b0375"}]: dispatch 2024-06-18T13:01:58.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717295cb-4786-41e4-9ac0-40fcea4b0375"}]: dispatch 2024-06-18T13:01:58.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "717295cb-4786-41e4-9ac0-40fcea4b0375"}]': finished 2024-06-18T13:01:58.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:57 smithi167 ceph-mon[37701]: osdmap e52: 8 total, 7 up, 8 in 2024-06-18T13:01:58.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:57 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:01:58.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:57 smithi167 ceph-mon[37701]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-18T13:01:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.179:0/1107809031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:01:59.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.179:0/1107809031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:01:59.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.179:0/1107809031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-18T13:02:00.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:01:59 smithi167 ceph-mon[29496]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-06-18T13:02:00.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:01:59 smithi167 ceph-mon[37701]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-06-18T13:02:00.262 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:01:59 smithi179 ceph-mon[33312]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-06-18T13:02:01.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:01 smithi179 ceph-mon[33312]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-06-18T13:02:01.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:01 smithi167 ceph-mon[29496]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-06-18T13:02:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:01 smithi167 ceph-mon[37701]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-06-18T13:02:03.882 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:03 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-18T13:02:03.883 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:03 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:03.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:03 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-18T13:02:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:03 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:03 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-18T13:02:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:03 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:04.847 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:04 smithi179 ceph-mon[33312]: Deploying daemon osd.7 on smithi179 2024-06-18T13:02:04.848 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:04 smithi179 ceph-mon[33312]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-18T13:02:04.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:04 smithi167 ceph-mon[29496]: Deploying daemon osd.7 on smithi179 2024-06-18T13:02:04.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:04 smithi167 ceph-mon[29496]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-18T13:02:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:04 smithi167 ceph-mon[37701]: Deploying daemon osd.7 on smithi179 2024-06-18T13:02:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:04 smithi167 ceph-mon[37701]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-18T13:02:05.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:05 smithi179 ceph-mon[33312]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-06-18T13:02:05.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:05 smithi167 ceph-mon[29496]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-06-18T13:02:05.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:05 smithi167 ceph-mon[37701]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-06-18T13:02:07.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:07 smithi179 ceph-mon[33312]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-06-18T13:02:07.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:07 smithi167 ceph-mon[29496]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-06-18T13:02:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:07 smithi167 ceph-mon[37701]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-06-18T13:02:08.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:08.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:08.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:08 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:08.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:08.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:08.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:08 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:08 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:09.885 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:09 smithi179 ceph-mon[33312]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-06-18T13:02:09.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:09 smithi167 ceph-mon[29496]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-06-18T13:02:09.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:09 smithi167 ceph-mon[37701]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-06-18T13:02:11.967 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 7 on host 'smithi179' 2024-06-18T13:02:11.967 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:11.967+0000 7ffb2b7fe640 1 -- 172.21.15.179:0/823725749 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffb34061de0 con 0x7ffb2006df50 2024-06-18T13:02:11.969 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:11.970+0000 7ffb38efa640 1 -- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ffb2006df50 msgr2=0x7ffb20070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:11.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:11.970+0000 7ffb38efa640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ffb2006df50 0x7ffb20070410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb240096c0 tx=0x7ffb24005fd0 comp rx=0 tx=0).stop 2024-06-18T13:02:11.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:11.970+0000 7ffb38efa640 1 -- 172.21.15.179:0/823725749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb34101e80 msgr2=0x7ffb3419e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:11.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:11.970+0000 7ffb38efa640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb34101e80 0x7ffb3419e170 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00b970 tx=0x7ffb2c00be30 comp rx=0 tx=0).stop 2024-06-18T13:02:11.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:11.970+0000 7ffb38efa640 1 -- 172.21.15.179:0/823725749 shutdown_connections 2024-06-18T13:02:11.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:11.970+0000 7ffb38efa640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7ffb2006df50 0x7ffb20070410 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:11.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:11.970+0000 7ffb38efa640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb34101e80 0x7ffb3419e170 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:11.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:11.970+0000 7ffb38efa640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb341014c0 0x7ffb3419bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:11.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:11.970+0000 7ffb38efa640 1 --2- 172.21.15.179:0/823725749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb341002c0 0x7ffb3406aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:11.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:11.970+0000 7ffb38efa640 1 -- 172.21.15.179:0/823725749 >> 172.21.15.179:0/823725749 conn(0x7ffb340fba50 msgr2=0x7ffb340fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:11.970 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:11.970+0000 7ffb38efa640 1 -- 172.21.15.179:0/823725749 shutdown_connections 2024-06-18T13:02:11.971 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:11.970+0000 7ffb38efa640 1 -- 172.21.15.179:0/823725749 wait complete. 2024-06-18T13:02:12.585 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:12 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.585 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:12 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.585 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:12 smithi179 ceph-mon[33312]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-06-18T13:02:12.585 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:12 smithi179 ceph-mon[33312]: from='osd.7 [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-18T13:02:12.585 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:12 smithi179 ceph-mon[33312]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-18T13:02:12.585 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:12 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:12.585 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:12 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:12.585 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:12 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.585 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:12 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:12.585 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:12 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.585 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:12 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.610 DEBUG:teuthology.orchestra.run.smithi179:osd.7> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.7.service 2024-06-18T13:02:12.613 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-06-18T13:02:12.613 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd stat -f json 2024-06-18T13:02:12.641 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.642 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.642 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[37701]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-06-18T13:02:12.642 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[37701]: from='osd.7 [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-18T13:02:12.642 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[37701]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-18T13:02:12.642 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:12.642 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:12.642 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.642 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:12.642 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.642 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[29496]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-06-18T13:02:12.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[29496]: from='osd.7 [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-18T13:02:12.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[29496]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-18T13:02:12.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:12.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:12.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.643 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:12.644 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.644 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:12 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:12.864 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:02:13.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:13 smithi179 ceph-mon[33312]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-18T13:02:13.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:13 smithi179 ceph-mon[33312]: from='osd.7 [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:02:13.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:13 smithi179 ceph-mon[33312]: osdmap e53: 8 total, 7 up, 8 in 2024-06-18T13:02:13.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:13 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:13.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:13 smithi179 ceph-mon[33312]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:02:13.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:13 smithi167 ceph-mon[29496]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-18T13:02:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:13 smithi167 ceph-mon[29496]: from='osd.7 [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:02:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:13 smithi167 ceph-mon[29496]: osdmap e53: 8 total, 7 up, 8 in 2024-06-18T13:02:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:13 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:13 smithi167 ceph-mon[29496]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:02:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:13 smithi167 ceph-mon[37701]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-18T13:02:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:13 smithi167 ceph-mon[37701]: from='osd.7 [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:02:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:13 smithi167 ceph-mon[37701]: osdmap e53: 8 total, 7 up, 8 in 2024-06-18T13:02:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:13 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:13 smithi167 ceph-mon[37701]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-18T13:02:13.957 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:02:13 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:02:13.700+0000 7f600d87c640 -1 osd.7 0 waiting for initial osdmap 2024-06-18T13:02:13.957 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:02:13 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:02:13.706+0000 7f6004e88640 -1 osd.7 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-18T13:02:14.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.374+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/1769473819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d81013b0 msgr2=0x7fc1d81017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:14.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.374+0000 7fc1dd7ab640 1 --2- 172.21.15.167:0/1769473819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d81013b0 0x7fc1d81017b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009860 tx=0x7fc1c802a340 comp rx=0 tx=0).stop 2024-06-18T13:02:14.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.374+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/1769473819 shutdown_connections 2024-06-18T13:02:14.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.374+0000 7fc1dd7ab640 1 --2- 172.21.15.167:0/1769473819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1d8102f70 0x7fc1d8109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:14.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.374+0000 7fc1dd7ab640 1 --2- 172.21.15.167:0/1769473819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d81025b0 0x7fc1d8102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:14.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.374+0000 7fc1dd7ab640 1 --2- 172.21.15.167:0/1769473819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d81013b0 0x7fc1d81017b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:14.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.374+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/1769473819 >> 172.21.15.167:0/1769473819 conn(0x7fc1d80fcb40 msgr2=0x7fc1d80fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:14.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.374+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/1769473819 shutdown_connections 2024-06-18T13:02:14.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.374+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/1769473819 wait complete. 2024-06-18T13:02:14.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.374+0000 7fc1dd7ab640 1 Processor -- start 2024-06-18T13:02:14.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1dd7ab640 1 -- start start 2024-06-18T13:02:14.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1dd7ab640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d81013b0 0x7fc1d8196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:14.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1dd7ab640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1d81025b0 0x7fc1d81971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:14.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1dd7ab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d8102f70 0x7fc1d819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:14.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1dd7ab640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d810c520 con 0x7fc1d8102f70 2024-06-18T13:02:14.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1dd7ab640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d810c3a0 con 0x7fc1d81025b0 2024-06-18T13:02:14.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1dd7ab640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d810c6a0 con 0x7fc1d81013b0 2024-06-18T13:02:14.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1d77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d8102f70 0x7fc1d819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:14.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1d6ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d81013b0 0x7fc1d8196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:14.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1d77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d8102f70 0x7fc1d819e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41572/0 (socket says 172.21.15.167:41572) 2024-06-18T13:02:14.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1d77fe640 1 -- 172.21.15.167:0/2483041444 learned_addr learned my addr 172.21.15.167:0/2483041444 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:02:14.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1d67fc640 1 --2- 172.21.15.167:0/2483041444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1d81025b0 0x7fc1d81971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:14.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1d6ffd640 1 -- 172.21.15.167:0/2483041444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1d81025b0 msgr2=0x7fc1d81971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:14.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1d6ffd640 1 --2- 172.21.15.167:0/2483041444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1d81025b0 0x7fc1d81971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:14.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1d6ffd640 1 -- 172.21.15.167:0/2483041444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d8102f70 msgr2=0x7fc1d819e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:14.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1d6ffd640 1 --2- 172.21.15.167:0/2483041444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d8102f70 0x7fc1d819e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:14.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1d6ffd640 1 -- 172.21.15.167:0/2483041444 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d819e890 con 0x7fc1d81013b0 2024-06-18T13:02:14.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1d77fe640 1 --2- 172.21.15.167:0/2483041444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d8102f70 0x7fc1d819e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:02:14.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.375+0000 7fc1d6ffd640 1 --2- 172.21.15.167:0/2483041444 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d81013b0 0x7fc1d8196c70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009780 tx=0x7fc1c8033d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:14.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.376+0000 7fc1bffff640 1 -- 172.21.15.167:0/2483041444 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8037040 con 0x7fc1d81013b0 2024-06-18T13:02:14.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.376+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/2483041444 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d819eb20 con 0x7fc1d81013b0 2024-06-18T13:02:14.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.376+0000 7fc1bffff640 1 -- 172.21.15.167:0/2483041444 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fc1c80307f0 con 0x7fc1d81013b0 2024-06-18T13:02:14.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.376+0000 7fc1bffff640 1 -- 172.21.15.167:0/2483041444 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80035f0 con 0x7fc1d81013b0 2024-06-18T13:02:14.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.376+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/2483041444 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d819efd0 con 0x7fc1d81013b0 2024-06-18T13:02:14.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.377+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/2483041444 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d81017b0 con 0x7fc1d81013b0 2024-06-18T13:02:14.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.377+0000 7fc1bffff640 1 -- 172.21.15.167:0/2483041444 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 15) ==== 92117+0+0 (secure 0 0 0) 0x7fc1c8042020 con 0x7fc1d81013b0 2024-06-18T13:02:14.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.378+0000 7fc1bffff640 1 --2- 172.21.15.167:0/2483041444 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc1c006df50 0x7fc1c0070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:14.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.378+0000 7fc1d67fc640 1 --2- 172.21.15.167:0/2483041444 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc1c006df50 0x7fc1c0070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:14.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.378+0000 7fc1bffff640 1 -- 172.21.15.167:0/2483041444 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(55..55 src has 1..55) ==== 5040+0+0 (secure 0 0 0) 0x7fc1c80acf60 con 0x7fc1d81013b0 2024-06-18T13:02:14.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.378+0000 7fc1d67fc640 1 --2- 172.21.15.167:0/2483041444 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc1c006df50 0x7fc1c0070410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009530 tx=0x7fc1cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:14.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.382+0000 7fc1bffff640 1 -- 172.21.15.167:0/2483041444 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fc1c8049020 con 0x7fc1d81013b0 2024-06-18T13:02:14.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[29496]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-18T13:02:14.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[29496]: osdmap e54: 8 total, 7 up, 8 in 2024-06-18T13:02:14.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:14.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:14.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[29496]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-06-18T13:02:14.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[29496]: osdmap e55: 8 total, 7 up, 8 in 2024-06-18T13:02:14.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:14.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[29496]: from='osd.7 ' entity='osd.7' 2024-06-18T13:02:14.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[37701]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-18T13:02:14.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[37701]: osdmap e54: 8 total, 7 up, 8 in 2024-06-18T13:02:14.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:14.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:14.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[37701]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-06-18T13:02:14.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[37701]: osdmap e55: 8 total, 7 up, 8 in 2024-06-18T13:02:14.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:14.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:14 smithi167 ceph-mon[37701]: from='osd.7 ' entity='osd.7' 2024-06-18T13:02:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.558+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/2483041444 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc1d80625a0 con 0x7fc1d81013b0 2024-06-18T13:02:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.559+0000 7fc1bffff640 1 -- 172.21.15.167:0/2483041444 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v55) ==== 74+0+130 (secure 0 0 0) 0x7fc1c807a660 con 0x7fc1d81013b0 2024-06-18T13:02:14.559 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T13:02:14.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.561+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/2483041444 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc1c006df50 msgr2=0x7fc1c0070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:14.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.561+0000 7fc1dd7ab640 1 --2- 172.21.15.167:0/2483041444 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc1c006df50 0x7fc1c0070410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009530 tx=0x7fc1cc009260 comp rx=0 tx=0).stop 2024-06-18T13:02:14.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.561+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/2483041444 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d81013b0 msgr2=0x7fc1d8196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:14.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.561+0000 7fc1dd7ab640 1 --2- 172.21.15.167:0/2483041444 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d81013b0 0x7fc1d8196c70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009780 tx=0x7fc1c8033d30 comp rx=0 tx=0).stop 2024-06-18T13:02:14.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.561+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/2483041444 shutdown_connections 2024-06-18T13:02:14.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.561+0000 7fc1dd7ab640 1 --2- 172.21.15.167:0/2483041444 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc1c006df50 0x7fc1c0070410 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:14.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.561+0000 7fc1dd7ab640 1 --2- 172.21.15.167:0/2483041444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d8102f70 0x7fc1d819e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:14.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.561+0000 7fc1dd7ab640 1 --2- 172.21.15.167:0/2483041444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1d81025b0 0x7fc1d81971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:14.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.561+0000 7fc1dd7ab640 1 --2- 172.21.15.167:0/2483041444 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1d81013b0 0x7fc1d8196c70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:14.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.561+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/2483041444 >> 172.21.15.167:0/2483041444 conn(0x7fc1d80fcb40 msgr2=0x7fc1d8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:14.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.561+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/2483041444 shutdown_connections 2024-06-18T13:02:14.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:14.561+0000 7fc1dd7ab640 1 -- 172.21.15.167:0/2483041444 wait complete. 2024-06-18T13:02:14.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:14 smithi179 ceph-mon[33312]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-18T13:02:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:14 smithi179 ceph-mon[33312]: osdmap e54: 8 total, 7 up, 8 in 2024-06-18T13:02:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:14 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:14 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:14 smithi179 ceph-mon[33312]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-06-18T13:02:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:14 smithi179 ceph-mon[33312]: osdmap e55: 8 total, 7 up, 8 in 2024-06-18T13:02:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:14 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:14 smithi179 ceph-mon[33312]: from='osd.7 ' entity='osd.7' 2024-06-18T13:02:15.315 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":55,"num_osds":8,"num_up_osds":7,"osd_up_since":1718715711,"num_in_osds":8,"osd_in_since":1718715717,"num_remapped_pgs":1} 2024-06-18T13:02:15.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:15 smithi167 ceph-mon[37701]: purged_snaps scrub starts 2024-06-18T13:02:15.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:15 smithi167 ceph-mon[37701]: purged_snaps scrub ok 2024-06-18T13:02:15.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:15 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:15.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2483041444' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-18T13:02:15.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:15 smithi167 ceph-mon[37701]: osd.7 [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249] boot 2024-06-18T13:02:15.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:15 smithi167 ceph-mon[37701]: osdmap e56: 8 total, 8 up, 8 in 2024-06-18T13:02:15.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:15 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:15.585 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:15 smithi167 ceph-mon[29496]: purged_snaps scrub starts 2024-06-18T13:02:15.585 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:15 smithi167 ceph-mon[29496]: purged_snaps scrub ok 2024-06-18T13:02:15.585 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:15 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:15.585 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2483041444' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-18T13:02:15.585 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:15 smithi167 ceph-mon[29496]: osd.7 [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249] boot 2024-06-18T13:02:15.585 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:15 smithi167 ceph-mon[29496]: osdmap e56: 8 total, 8 up, 8 in 2024-06-18T13:02:15.585 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:15 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:15.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:15 smithi179 ceph-mon[33312]: purged_snaps scrub starts 2024-06-18T13:02:15.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:15 smithi179 ceph-mon[33312]: purged_snaps scrub ok 2024-06-18T13:02:15.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:15 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:15.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2483041444' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-18T13:02:15.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:15 smithi179 ceph-mon[33312]: osd.7 [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249] boot 2024-06-18T13:02:15.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:15 smithi179 ceph-mon[33312]: osdmap e56: 8 total, 8 up, 8 in 2024-06-18T13:02:15.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:15 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:02:16.316 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd stat -f json 2024-06-18T13:02:16.563 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:02:16.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:16 smithi179 ceph-mon[33312]: Detected new or changed devices on smithi179 2024-06-18T13:02:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:16 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:16 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:16 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:02:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:16 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:02:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:16 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:02:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:16 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:02:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:16 smithi179 ceph-mon[33312]: Adjusting osd_memory_target on smithi179 to 4272M 2024-06-18T13:02:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:16 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:16 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:16 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:16 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:16 smithi179 ceph-mon[33312]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:02:16.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:16 smithi179 ceph-mon[33312]: osdmap e57: 8 total, 8 up, 8 in 2024-06-18T13:02:16.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[37701]: Detected new or changed devices on smithi179 2024-06-18T13:02:16.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:16.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:16.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:02:16.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:02:16.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:02:16.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:02:16.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[37701]: Adjusting osd_memory_target on smithi179 to 4272M 2024-06-18T13:02:16.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:16.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:16.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:16.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:16.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[37701]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:02:16.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[37701]: osdmap e57: 8 total, 8 up, 8 in 2024-06-18T13:02:16.686 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[29496]: Detected new or changed devices on smithi179 2024-06-18T13:02:16.686 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:16.686 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:16.687 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:02:16.687 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:02:16.687 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:02:16.687 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:02:16.687 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[29496]: Adjusting osd_memory_target on smithi179 to 4272M 2024-06-18T13:02:16.687 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:16.687 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:16.687 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:16.687 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:16.687 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[29496]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:02:16.687 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:16 smithi167 ceph-mon[29496]: osdmap e57: 8 total, 8 up, 8 in 2024-06-18T13:02:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:17 smithi179 ceph-mon[33312]: osdmap e58: 8 total, 8 up, 8 in 2024-06-18T13:02:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:17 smithi179 ceph-mon[33312]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:02:17.941 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:17 smithi167 ceph-mon[29496]: osdmap e58: 8 total, 8 up, 8 in 2024-06-18T13:02:17.941 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:17 smithi167 ceph-mon[29496]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:02:17.941 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:17 smithi167 ceph-mon[37701]: osdmap e58: 8 total, 8 up, 8 in 2024-06-18T13:02:17.941 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:17 smithi167 ceph-mon[37701]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:02:18.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.215+0000 7fb0a0d24640 1 -- 172.21.15.167:0/2931907975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb09c1002c0 msgr2=0x7fb09c1006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:18.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.215+0000 7fb0a0d24640 1 --2- 172.21.15.167:0/2931907975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb09c1002c0 0x7fb09c1006c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009900 tx=0x7fb08c02a3f0 comp rx=0 tx=0).stop 2024-06-18T13:02:18.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.215+0000 7fb0a0d24640 1 -- 172.21.15.167:0/2931907975 shutdown_connections 2024-06-18T13:02:18.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.215+0000 7fb0a0d24640 1 --2- 172.21.15.167:0/2931907975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c101e80 0x7fb09c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:18.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.215+0000 7fb0a0d24640 1 --2- 172.21.15.167:0/2931907975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb09c1014c0 0x7fb09c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:18.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.215+0000 7fb0a0d24640 1 --2- 172.21.15.167:0/2931907975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb09c1002c0 0x7fb09c1006c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:18.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.215+0000 7fb0a0d24640 1 -- 172.21.15.167:0/2931907975 >> 172.21.15.167:0/2931907975 conn(0x7fb09c0fba50 msgr2=0x7fb09c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:18.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.215+0000 7fb0a0d24640 1 -- 172.21.15.167:0/2931907975 shutdown_connections 2024-06-18T13:02:18.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.215+0000 7fb0a0d24640 1 -- 172.21.15.167:0/2931907975 wait complete. 2024-06-18T13:02:18.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.216+0000 7fb0a0d24640 1 Processor -- start 2024-06-18T13:02:18.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.216+0000 7fb0a0d24640 1 -- start start 2024-06-18T13:02:18.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.216+0000 7fb0a0d24640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb09c1002c0 0x7fb09c104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:18.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.216+0000 7fb0a0d24640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb09c1014c0 0x7fb09c1028a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:18.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.216+0000 7fb0a0d24640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c101e80 0x7fb09c102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:18.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.216+0000 7fb0a0d24640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c10b430 con 0x7fb09c101e80 2024-06-18T13:02:18.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.216+0000 7fb0a0d24640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c10b2b0 con 0x7fb09c1014c0 2024-06-18T13:02:18.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.216+0000 7fb0a0d24640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb09c10b5b0 con 0x7fb09c1002c0 2024-06-18T13:02:18.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb09a59c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb09c1002c0 0x7fb09c104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:18.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb09ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c101e80 0x7fb09c102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:18.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb09a59c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb09c1002c0 0x7fb09c104250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:44272/0 (socket says 172.21.15.167:44272) 2024-06-18T13:02:18.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb09a59c640 1 -- 172.21.15.167:0/1636345163 learned_addr learned my addr 172.21.15.167:0/1636345163 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:02:18.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb099d9b640 1 --2- 172.21.15.167:0/1636345163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb09c1014c0 0x7fb09c1028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:18.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb09a59c640 1 -- 172.21.15.167:0/1636345163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb09c1014c0 msgr2=0x7fb09c1028a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:18.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb09a59c640 1 --2- 172.21.15.167:0/1636345163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb09c1014c0 0x7fb09c1028a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:18.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb09a59c640 1 -- 172.21.15.167:0/1636345163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c101e80 msgr2=0x7fb09c102de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:18.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb09a59c640 1 --2- 172.21.15.167:0/1636345163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c101e80 0x7fb09c102de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:18.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb09a59c640 1 -- 172.21.15.167:0/1636345163 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c1036a0 con 0x7fb09c1002c0 2024-06-18T13:02:18.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb09ad9d640 1 --2- 172.21.15.167:0/1636345163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c101e80 0x7fb09c102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:02:18.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb09a59c640 1 --2- 172.21.15.167:0/1636345163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb09c1002c0 0x7fb09c104250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb08c005940 tx=0x7fb08c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:18.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb08b7fe640 1 -- 172.21.15.167:0/1636345163 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c03f070 con 0x7fb09c1002c0 2024-06-18T13:02:18.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb0a0d24640 1 -- 172.21.15.167:0/1636345163 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09c1a0e80 con 0x7fb09c1002c0 2024-06-18T13:02:18.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb08b7fe640 1 -- 172.21.15.167:0/1636345163 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fb08c003a70 con 0x7fb09c1002c0 2024-06-18T13:02:18.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb08b7fe640 1 -- 172.21.15.167:0/1636345163 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c035030 con 0x7fb09c1002c0 2024-06-18T13:02:18.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.217+0000 7fb0a0d24640 1 -- 172.21.15.167:0/1636345163 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb09c1a12a0 con 0x7fb09c1002c0 2024-06-18T13:02:18.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.218+0000 7fb0a0d24640 1 -- 172.21.15.167:0/1636345163 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb09c1006c0 con 0x7fb09c1002c0 2024-06-18T13:02:18.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.219+0000 7fb08b7fe640 1 -- 172.21.15.167:0/1636345163 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 15) ==== 92117+0+0 (secure 0 0 0) 0x7fb08c042070 con 0x7fb09c1002c0 2024-06-18T13:02:18.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.220+0000 7fb08b7fe640 1 --2- 172.21.15.167:0/1636345163 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fb08406df50 0x7fb084070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:18.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.220+0000 7fb099d9b640 1 --2- 172.21.15.167:0/1636345163 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fb08406df50 0x7fb084070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:18.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.220+0000 7fb08b7fe640 1 -- 172.21.15.167:0/1636345163 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(59..59 src has 1..59) ==== 5287+0+0 (secure 0 0 0) 0x7fb08c0ae5b0 con 0x7fb09c1002c0 2024-06-18T13:02:18.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.220+0000 7fb099d9b640 1 --2- 172.21.15.167:0/1636345163 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fb08406df50 0x7fb084070410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb09c103fd0 tx=0x7fb090008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:18.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.224+0000 7fb08b7fe640 1 -- 172.21.15.167:0/1636345163 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fb08c07c200 con 0x7fb09c1002c0 2024-06-18T13:02:18.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.398+0000 7fb0a0d24640 1 -- 172.21.15.167:0/1636345163 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb09c0625a0 con 0x7fb09c1002c0 2024-06-18T13:02:18.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.398+0000 7fb08b7fe640 1 -- 172.21.15.167:0/1636345163 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v59) ==== 74+0+130 (secure 0 0 0) 0x7fb08c07bb00 con 0x7fb09c1002c0 2024-06-18T13:02:18.398 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T13:02:18.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.400+0000 7fb0a0d24640 1 -- 172.21.15.167:0/1636345163 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fb08406df50 msgr2=0x7fb084070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:18.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.400+0000 7fb0a0d24640 1 --2- 172.21.15.167:0/1636345163 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fb08406df50 0x7fb084070410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb09c103fd0 tx=0x7fb090008040 comp rx=0 tx=0).stop 2024-06-18T13:02:18.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.400+0000 7fb0a0d24640 1 -- 172.21.15.167:0/1636345163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb09c1002c0 msgr2=0x7fb09c104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:18.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.400+0000 7fb0a0d24640 1 --2- 172.21.15.167:0/1636345163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb09c1002c0 0x7fb09c104250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb08c005940 tx=0x7fb08c005dc0 comp rx=0 tx=0).stop 2024-06-18T13:02:18.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.401+0000 7fb0a0d24640 1 -- 172.21.15.167:0/1636345163 shutdown_connections 2024-06-18T13:02:18.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.401+0000 7fb0a0d24640 1 --2- 172.21.15.167:0/1636345163 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fb08406df50 0x7fb084070410 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:18.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.401+0000 7fb0a0d24640 1 --2- 172.21.15.167:0/1636345163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c101e80 0x7fb09c102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:18.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.401+0000 7fb0a0d24640 1 --2- 172.21.15.167:0/1636345163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb09c1014c0 0x7fb09c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:18.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.401+0000 7fb0a0d24640 1 --2- 172.21.15.167:0/1636345163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb09c1002c0 0x7fb09c104250 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:18.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.401+0000 7fb0a0d24640 1 -- 172.21.15.167:0/1636345163 >> 172.21.15.167:0/1636345163 conn(0x7fb09c0fba50 msgr2=0x7fb09c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:18.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.401+0000 7fb0a0d24640 1 -- 172.21.15.167:0/1636345163 shutdown_connections 2024-06-18T13:02:18.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:18.401+0000 7fb0a0d24640 1 -- 172.21.15.167:0/1636345163 wait complete. 2024-06-18T13:02:18.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:18 smithi179 ceph-mon[33312]: osdmap e59: 8 total, 8 up, 8 in 2024-06-18T13:02:18.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1636345163' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-18T13:02:18.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:18 smithi167 ceph-mon[29496]: osdmap e59: 8 total, 8 up, 8 in 2024-06-18T13:02:18.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1636345163' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-18T13:02:18.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:18 smithi167 ceph-mon[37701]: osdmap e59: 8 total, 8 up, 8 in 2024-06-18T13:02:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1636345163' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-18T13:02:19.040 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":59,"num_osds":8,"num_up_osds":8,"osd_up_since":1718715734,"num_in_osds":8,"osd_in_since":1718715717,"num_remapped_pgs":0} 2024-06-18T13:02:19.041 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd dump --format=json 2024-06-18T13:02:19.277 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:02:19.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:19 smithi167 ceph-mon[37701]: osdmap e60: 8 total, 8 up, 8 in 2024-06-18T13:02:19.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:19 smithi167 ceph-mon[37701]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:02:19.703 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:19 smithi167 ceph-mon[29496]: osdmap e60: 8 total, 8 up, 8 in 2024-06-18T13:02:19.703 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:19 smithi167 ceph-mon[29496]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:02:20.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:19 smithi179 ceph-mon[33312]: osdmap e60: 8 total, 8 up, 8 in 2024-06-18T13:02:20.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:19 smithi179 ceph-mon[33312]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:02:20.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.949+0000 7fcb49324640 1 -- 172.21.15.167:0/1835438520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb441055c0 msgr2=0x7fcb4410d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:20.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.949+0000 7fcb49324640 1 --2- 172.21.15.167:0/1835438520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb441055c0 0x7fcb4410d140 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009be0 tx=0x7fcb3c02a6c0 comp rx=0 tx=0).stop 2024-06-18T13:02:20.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.949+0000 7fcb49324640 1 -- 172.21.15.167:0/1835438520 shutdown_connections 2024-06-18T13:02:20.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.949+0000 7fcb49324640 1 --2- 172.21.15.167:0/1835438520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb441055c0 0x7fcb4410d140 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:20.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.949+0000 7fcb49324640 1 --2- 172.21.15.167:0/1835438520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb44100e40 0x7fcb44104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:20.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.949+0000 7fcb49324640 1 --2- 172.21.15.167:0/1835438520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb44100490 0x7fcb44100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:20.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.949+0000 7fcb49324640 1 -- 172.21.15.167:0/1835438520 >> 172.21.15.167:0/1835438520 conn(0x7fcb440fbf50 msgr2=0x7fcb440fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:20.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.949+0000 7fcb49324640 1 -- 172.21.15.167:0/1835438520 shutdown_connections 2024-06-18T13:02:20.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.949+0000 7fcb49324640 1 -- 172.21.15.167:0/1835438520 wait complete. 2024-06-18T13:02:20.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.949+0000 7fcb49324640 1 Processor -- start 2024-06-18T13:02:20.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb49324640 1 -- start start 2024-06-18T13:02:20.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb49324640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb44100490 0x7fcb440fc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:20.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb49324640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb44100e40 0x7fcb440fcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:20.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb49324640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb441055c0 0x7fcb440fd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:20.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb49324640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4410faf0 con 0x7fcb44100e40 2024-06-18T13:02:20.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb49324640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4410f970 con 0x7fcb441055c0 2024-06-18T13:02:20.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb49324640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4410fc70 con 0x7fcb44100490 2024-06-18T13:02:20.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb42ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb44100490 0x7fcb440fc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:20.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb427fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb44100e40 0x7fcb440fcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:20.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb42ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb44100490 0x7fcb440fc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:44296/0 (socket says 172.21.15.167:44296) 2024-06-18T13:02:20.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb42ffd640 1 -- 172.21.15.167:0/2665443057 learned_addr learned my addr 172.21.15.167:0/2665443057 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:02:20.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb437fe640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb441055c0 0x7fcb440fd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:20.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb427fc640 1 -- 172.21.15.167:0/2665443057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb44100490 msgr2=0x7fcb440fc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:20.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb427fc640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb44100490 0x7fcb440fc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:20.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb427fc640 1 -- 172.21.15.167:0/2665443057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb441055c0 msgr2=0x7fcb440fd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:20.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb427fc640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb441055c0 0x7fcb440fd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:20.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb427fc640 1 -- 172.21.15.167:0/2665443057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb441a55d0 con 0x7fcb44100e40 2024-06-18T13:02:20.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb42ffd640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb44100490 0x7fcb440fc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:02:20.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb437fe640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb441055c0 0x7fcb440fd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T13:02:20.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.950+0000 7fcb427fc640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb44100e40 0x7fcb440fcb00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400b3f0 tx=0x7fcb3400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:20.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.951+0000 7fcb2bfff640 1 -- 172.21.15.167:0/2665443057 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34015020 con 0x7fcb44100e40 2024-06-18T13:02:20.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.951+0000 7fcb2bfff640 1 -- 172.21.15.167:0/2665443057 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fcb34011cb0 con 0x7fcb44100e40 2024-06-18T13:02:20.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.951+0000 7fcb49324640 1 -- 172.21.15.167:0/2665443057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb441a58c0 con 0x7fcb44100e40 2024-06-18T13:02:20.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.951+0000 7fcb2bfff640 1 -- 172.21.15.167:0/2665443057 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34010c40 con 0x7fcb44100e40 2024-06-18T13:02:20.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.951+0000 7fcb49324640 1 -- 172.21.15.167:0/2665443057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb441a5db0 con 0x7fcb44100e40 2024-06-18T13:02:20.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.952+0000 7fcb49324640 1 -- 172.21.15.167:0/2665443057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb44100870 con 0x7fcb44100e40 2024-06-18T13:02:20.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.952+0000 7fcb2bfff640 1 -- 172.21.15.167:0/2665443057 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 15) ==== 92117+0+0 (secure 0 0 0) 0x7fcb34018020 con 0x7fcb44100e40 2024-06-18T13:02:20.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.953+0000 7fcb2bfff640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcb2c06df50 0x7fcb2c070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:20.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.953+0000 7fcb2bfff640 1 -- 172.21.15.167:0/2665443057 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5287+0+0 (secure 0 0 0) 0x7fcb3408d4b0 con 0x7fcb44100e40 2024-06-18T13:02:20.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.953+0000 7fcb42ffd640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcb2c06df50 0x7fcb2c070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:20.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.953+0000 7fcb42ffd640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcb2c06df50 0x7fcb2c070410 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb380096c0 tx=0x7fcb38009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:20.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:20.957+0000 7fcb2bfff640 1 -- 172.21.15.167:0/2665443057 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fcb3401f020 con 0x7fcb44100e40 2024-06-18T13:02:21.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:21.133+0000 7fcb49324640 1 -- 172.21.15.167:0/2665443057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb441a6010 con 0x7fcb44100e40 2024-06-18T13:02:21.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:21.134+0000 7fcb2bfff640 1 -- 172.21.15.167:0/2665443057 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+13613 (secure 0 0 0) 0x7fcb3405b100 con 0x7fcb44100e40 2024-06-18T13:02:21.134 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T13:02:21.134 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":60,"fsid":"3434f1e2-2d72-11ef-bc9d-c7b262605968","created":"2024-06-18T12:57:18.945902+0000","modified":"2024-06-18T13:02:18.630886+0000","last_up_change":"2024-06-18T13:02:14.621219+0000","last_in_change":"2024-06-18T13:01:57.489323+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-18T13:00:21.794480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df01b396-984c-4e66-83a8-e439554c688a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3001811419},{"type":"v1","addr":"172.21.15.167:6803","nonce":3001811419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3001811419},{"type":"v1","addr":"172.21.15.167:6805","nonce":3001811419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3001811419},{"type":"v1","addr":"172.21.15.167:6809","nonce":3001811419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3001811419},{"type":"v1","addr":"172.21.15.167:6807","nonce":3001811419}]},"public_addr":"172.21.15.167:6803/3001811419","cluster_addr":"172.21.15.167:6805/3001811419","heartbeat_back_addr":"172.21.15.167:6809/3001811419","heartbeat_front_addr":"172.21.15.167:6807/3001811419","state":["exists","up"]},{"osd":1,"uuid":"d8159aff-9414-4ebe-897f-ffbf27ed9e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2010202972},{"type":"v1","addr":"172.21.15.167:6811","nonce":2010202972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2010202972},{"type":"v1","addr":"172.21.15.167:6813","nonce":2010202972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2010202972},{"type":"v1","addr":"172.21.15.167:6817","nonce":2010202972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2010202972},{"type":"v1","addr":"172.21.15.167:6815","nonce":2010202972}]},"public_addr":"172.21.15.167:6811/2010202972","cluster_addr":"172.21.15.167:6813/2010202972","heartbeat_back_addr":"172.21.15.167:6817/2010202972","heartbeat_front_addr":"172.21.15.167:6815/2010202972","state":["exists","up"]},{"osd":2,"uuid":"81482f44-3d16-4c10-b2ac-88eeeff554fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":183062921},{"type":"v1","addr":"172.21.15.167:6819","nonce":183062921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":183062921},{"type":"v1","addr":"172.21.15.167:6821","nonce":183062921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":183062921},{"type":"v1","addr":"172.21.15.167:6825","nonce":183062921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":183062921},{"type":"v1","addr":"172.21.15.167:6823","nonce":183062921}]},"public_addr":"172.21.15.167:6819/183062921","cluster_addr":"172.21.15.167:6821/183062921","heartbeat_back_addr":"172.21.15.167:6825/183062921","heartbeat_front_addr":"172.21.15.167:6823/183062921","state":["exists","up"]},{"osd":3,"uuid":"5958775e-faaa-4fba-9b33-dcd46c049926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":2828000547},{"type":"v1","addr":"172.21.15.167:6827","nonce":2828000547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":2828000547},{"type":"v1","addr":"172.21.15.167:6829","nonce":2828000547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828000547},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828000547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":2828000547},{"type":"v1","addr":"172.21.15.167:6831","nonce":2828000547}]},"public_addr":"172.21.15.167:6827/2828000547","cluster_addr":"172.21.15.167:6829/2828000547","heartbeat_back_addr":"172.21.15.167:6833/2828000547","heartbeat_front_addr":"172.21.15.167:6831/2828000547","state":["exists","up"]},{"osd":4,"uuid":"6abe43b9-eb57-4e7c-afce-fa7ff1b829c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3925771696},{"type":"v1","addr":"172.21.15.179:6801","nonce":3925771696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3925771696},{"type":"v1","addr":"172.21.15.179:6803","nonce":3925771696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3925771696},{"type":"v1","addr":"172.21.15.179:6807","nonce":3925771696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3925771696},{"type":"v1","addr":"172.21.15.179:6805","nonce":3925771696}]},"public_addr":"172.21.15.179:6801/3925771696","cluster_addr":"172.21.15.179:6803/3925771696","heartbeat_back_addr":"172.21.15.179:6807/3925771696","heartbeat_front_addr":"172.21.15.179:6805/3925771696","state":["exists","up"]},{"osd":5,"uuid":"45d6f34f-ce3b-4465-baa2-b49aec949910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1550696992},{"type":"v1","addr":"172.21.15.179:6809","nonce":1550696992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1550696992},{"type":"v1","addr":"172.21.15.179:6811","nonce":1550696992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1550696992},{"type":"v1","addr":"172.21.15.179:6815","nonce":1550696992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1550696992},{"type":"v1","addr":"172.21.15.179:6813","nonce":1550696992}]},"public_addr":"172.21.15.179:6809/1550696992","cluster_addr":"172.21.15.179:6811/1550696992","heartbeat_back_addr":"172.21.15.179:6815/1550696992","heartbeat_front_addr":"172.21.15.179:6813/1550696992","state":["exists","up"]},{"osd":6,"uuid":"06a47b6b-eac6-440b-9037-b55bfaf29e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":153587065},{"type":"v1","addr":"172.21.15.179:6817","nonce":153587065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":153587065},{"type":"v1","addr":"172.21.15.179:6819","nonce":153587065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":153587065},{"type":"v1","addr":"172.21.15.179:6823","nonce":153587065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":153587065},{"type":"v1","addr":"172.21.15.179:6821","nonce":153587065}]},"public_addr":"172.21.15.179:6817/153587065","cluster_addr":"172.21.15.179:6819/153587065","heartbeat_back_addr":"172.21.15.179:6823/153587065","heartbeat_front_addr":"172.21.15.179:6821/153587065","state":["exists","up"]},{"osd":7,"uuid":"717295cb-4786-41e4-9ac0-40fcea4b0375","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":3253527249},{"type":"v1","addr":"172.21.15.179:6825","nonce":3253527249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3253527249},{"type":"v1","addr":"172.21.15.179:6827","nonce":3253527249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":3253527249},{"type":"v1","addr":"172.21.15.179:6831","nonce":3253527249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":3253527249},{"type":"v1","addr":"172.21.15.179:6829","nonce":3253527249}]},"public_addr":"172.21.15.179:6825/3253527249","cluster_addr":"172.21.15.179:6827/3253527249","heartbeat_back_addr":"172.21.15.179:6831/3253527249","heartbeat_front_addr":"172.21.15.179:6829/3253527249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T12:59:31.344584+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T12:59:54.440221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:00:18.463935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:00:42.274566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:01:03.689366+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:01:26.918997+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:01:50.098743+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:02:12.629691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:0/2519223545":"2024-06-19T12:58:13.490367+0000","172.21.15.167:0/3211351643":"2024-06-19T12:58:13.490367+0000","172.21.15.167:6801/1873453520":"2024-06-19T12:58:13.490367+0000","172.21.15.167:6800/1873453520":"2024-06-19T12:58:13.490367+0000","172.21.15.167:0/3011148832":"2024-06-19T12:58:13.490367+0000","172.21.15.167:0/3606279536":"2024-06-19T12:57:51.125515+0000","172.21.15.167:0/999980396":"2024-06-19T12:57:51.125515+0000","172.21.15.167:6801/2756279078":"2024-06-19T12:57:51.125515+0000","172.21.15.167:0/1156304959":"2024-06-19T12:57:51.125515+0000","172.21.15.167:6800/2756279078":"2024-06-19T12:57:51.125515+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-18T13:02:21.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:21.136+0000 7fcb49324640 1 -- 172.21.15.167:0/2665443057 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcb2c06df50 msgr2=0x7fcb2c070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:21.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:21.136+0000 7fcb49324640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcb2c06df50 0x7fcb2c070410 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb380096c0 tx=0x7fcb38009260 comp rx=0 tx=0).stop 2024-06-18T13:02:21.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:21.136+0000 7fcb49324640 1 -- 172.21.15.167:0/2665443057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb44100e40 msgr2=0x7fcb440fcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:21.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:21.136+0000 7fcb49324640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb44100e40 0x7fcb440fcb00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400b3f0 tx=0x7fcb3400b8b0 comp rx=0 tx=0).stop 2024-06-18T13:02:21.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:21.136+0000 7fcb49324640 1 -- 172.21.15.167:0/2665443057 shutdown_connections 2024-06-18T13:02:21.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:21.136+0000 7fcb49324640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcb2c06df50 0x7fcb2c070410 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:21.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:21.136+0000 7fcb49324640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb441055c0 0x7fcb440fd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:21.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:21.136+0000 7fcb49324640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb44100e40 0x7fcb440fcb00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:21.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:21.136+0000 7fcb49324640 1 --2- 172.21.15.167:0/2665443057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb44100490 0x7fcb440fc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:21.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:21.136+0000 7fcb49324640 1 -- 172.21.15.167:0/2665443057 >> 172.21.15.167:0/2665443057 conn(0x7fcb440fbf50 msgr2=0x7fcb441042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:21.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:21.137+0000 7fcb49324640 1 -- 172.21.15.167:0/2665443057 shutdown_connections 2024-06-18T13:02:21.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:21.137+0000 7fcb49324640 1 -- 172.21.15.167:0/2665443057 wait complete. 2024-06-18T13:02:21.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2665443057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-18T13:02:21.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2665443057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-18T13:02:21.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2665443057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-18T13:02:21.866 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-18T13:00:21.794480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-18T13:02:21.867 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-18T13:02:22.104 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:02:22.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:22 smithi167 ceph-mon[37701]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-18T13:02:22.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:22 smithi167 ceph-mon[29496]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-18T13:02:22.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:22 smithi179 ceph-mon[33312]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-18T13:02:23.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.583+0000 7f0d1f947640 1 -- 172.21.15.167:0/1632027473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d181066c0 msgr2=0x7f0d1810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:23.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.583+0000 7f0d1f947640 1 --2- 172.21.15.167:0/1632027473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d181066c0 0x7f0d1810e240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009be0 tx=0x7f0d1002a6c0 comp rx=0 tx=0).stop 2024-06-18T13:02:23.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.583+0000 7f0d1f947640 1 -- 172.21.15.167:0/1632027473 shutdown_connections 2024-06-18T13:02:23.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.583+0000 7f0d1f947640 1 --2- 172.21.15.167:0/1632027473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d181066c0 0x7f0d1810e240 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:23.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.583+0000 7f0d1f947640 1 --2- 172.21.15.167:0/1632027473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d18101f40 0x7f0d18105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:23.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.583+0000 7f0d1f947640 1 --2- 172.21.15.167:0/1632027473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d18101590 0x7f0d18101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:23.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.583+0000 7f0d1f947640 1 -- 172.21.15.167:0/1632027473 >> 172.21.15.167:0/1632027473 conn(0x7f0d180fd070 msgr2=0x7f0d180ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:23.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.583+0000 7f0d1f947640 1 -- 172.21.15.167:0/1632027473 shutdown_connections 2024-06-18T13:02:23.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.583+0000 7f0d1f947640 1 -- 172.21.15.167:0/1632027473 wait complete. 2024-06-18T13:02:23.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.584+0000 7f0d1f947640 1 Processor -- start 2024-06-18T13:02:23.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.584+0000 7f0d1f947640 1 -- start start 2024-06-18T13:02:23.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.584+0000 7f0d1f947640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d18101590 0x7f0d18197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:23.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.584+0000 7f0d1f947640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d18101f40 0x7f0d181976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:23.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.584+0000 7f0d1f947640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d181066c0 0x7f0d1819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:23.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.584+0000 7f0d1f947640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d18110bb0 con 0x7f0d18101590 2024-06-18T13:02:23.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.584+0000 7f0d1f947640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0d18110a30 con 0x7f0d181066c0 2024-06-18T13:02:23.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.584+0000 7f0d1f947640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0d18110d30 con 0x7f0d18101f40 2024-06-18T13:02:23.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.584+0000 7f0d1d6e3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d18101590 0x7f0d18197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:23.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.584+0000 7f0d1cee2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d18101f40 0x7f0d181976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:23.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.584+0000 7f0d1d6e3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d18101590 0x7f0d18197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45110/0 (socket says 172.21.15.167:45110) 2024-06-18T13:02:23.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.584+0000 7f0d1d6e3640 1 -- 172.21.15.167:0/2267958020 learned_addr learned my addr 172.21.15.167:0/2267958020 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:02:23.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.584+0000 7f0d1cee2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d18101f40 0x7f0d181976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:34318/0 (socket says 172.21.15.167:34318) 2024-06-18T13:02:23.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.585+0000 7f0d1dee4640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d181066c0 0x7f0d1819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:23.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.585+0000 7f0d1cee2640 1 -- 172.21.15.167:0/2267958020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d181066c0 msgr2=0x7f0d1819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:23.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.585+0000 7f0d1cee2640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d181066c0 0x7f0d1819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:23.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.585+0000 7f0d1cee2640 1 -- 172.21.15.167:0/2267958020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d18101590 msgr2=0x7f0d18197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:23.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.585+0000 7f0d1cee2640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d18101590 0x7f0d18197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:23.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.585+0000 7f0d1cee2640 1 -- 172.21.15.167:0/2267958020 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1819c130 con 0x7f0d18101f40 2024-06-18T13:02:23.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.585+0000 7f0d1dee4640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d181066c0 0x7f0d1819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-18T13:02:23.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.585+0000 7f0d1d6e3640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d18101590 0x7f0d18197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:02:23.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.585+0000 7f0d1cee2640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d18101f40 0x7f0d181976c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0d08005fd0 tx=0x7f0d0800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:23.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.585+0000 7f0d0e7fc640 1 -- 172.21.15.167:0/2267958020 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0800cd00 con 0x7f0d18101f40 2024-06-18T13:02:23.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.585+0000 7f0d0e7fc640 1 -- 172.21.15.167:0/2267958020 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f0d080035f0 con 0x7f0d18101f40 2024-06-18T13:02:23.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.585+0000 7f0d1f947640 1 -- 172.21.15.167:0/2267958020 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1819c420 con 0x7f0d18101f40 2024-06-18T13:02:23.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.585+0000 7f0d0e7fc640 1 -- 172.21.15.167:0/2267958020 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d080044a0 con 0x7f0d18101f40 2024-06-18T13:02:23.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.585+0000 7f0d1f947640 1 -- 172.21.15.167:0/2267958020 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d181a3d00 con 0x7f0d18101f40 2024-06-18T13:02:23.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.586+0000 7f0d1f947640 1 -- 172.21.15.167:0/2267958020 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d18101970 con 0x7f0d18101f40 2024-06-18T13:02:23.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.586+0000 7f0d0e7fc640 1 -- 172.21.15.167:0/2267958020 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 15) ==== 92117+0+0 (secure 0 0 0) 0x7f0d08004000 con 0x7f0d18101f40 2024-06-18T13:02:23.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.587+0000 7f0d0e7fc640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f0d0406df50 0x7f0d04070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:23.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.587+0000 7f0d0e7fc640 1 -- 172.21.15.167:0/2267958020 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(60..60 src has 1..60) ==== 5287+0+0 (secure 0 0 0) 0x7f0d0808dee0 con 0x7f0d18101f40 2024-06-18T13:02:23.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.587+0000 7f0d1d6e3640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f0d0406df50 0x7f0d04070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:23.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.587+0000 7f0d1d6e3640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f0d0406df50 0x7f0d04070410 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009e10 tx=0x7f0d140091c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:23.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.593+0000 7f0d0e7fc640 1 -- 172.21.15.167:0/2267958020 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f0d0805bb30 con 0x7f0d18101f40 2024-06-18T13:02:23.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:23 smithi167 ceph-mon[37701]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-18T13:02:23.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:23 smithi167 ceph-mon[29496]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-18T13:02:23.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.771+0000 7f0d1f947640 1 -- 172.21.15.167:0/2267958020 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d181a3f30 con 0x7f0d18101f40 2024-06-18T13:02:23.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.772+0000 7f0d0e7fc640 1 -- 172.21.15.167:0/2267958020 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) ==== 93+0+10 (secure 0 0 0) 0x7f0d0805b430 con 0x7f0d18101f40 2024-06-18T13:02:23.771 INFO:teuthology.orchestra.run.smithi167.stdout:pg_num: 1 2024-06-18T13:02:23.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.774+0000 7f0d1f947640 1 -- 172.21.15.167:0/2267958020 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f0d0406df50 msgr2=0x7f0d04070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:23.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.774+0000 7f0d1f947640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f0d0406df50 0x7f0d04070410 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009e10 tx=0x7f0d140091c0 comp rx=0 tx=0).stop 2024-06-18T13:02:23.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.774+0000 7f0d1f947640 1 -- 172.21.15.167:0/2267958020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d18101f40 msgr2=0x7f0d181976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:23.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.774+0000 7f0d1f947640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d18101f40 0x7f0d181976c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0d08005fd0 tx=0x7f0d0800ee50 comp rx=0 tx=0).stop 2024-06-18T13:02:23.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.774+0000 7f0d1f947640 1 -- 172.21.15.167:0/2267958020 shutdown_connections 2024-06-18T13:02:23.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.774+0000 7f0d1f947640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f0d0406df50 0x7f0d04070410 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:23.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.774+0000 7f0d1f947640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d181066c0 0x7f0d1819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:23.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.774+0000 7f0d1f947640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d18101f40 0x7f0d181976c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:23.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.774+0000 7f0d1f947640 1 --2- 172.21.15.167:0/2267958020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d18101590 0x7f0d18197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:23.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.774+0000 7f0d1f947640 1 -- 172.21.15.167:0/2267958020 >> 172.21.15.167:0/2267958020 conn(0x7f0d180fd070 msgr2=0x7f0d180fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:23.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.774+0000 7f0d1f947640 1 -- 172.21.15.167:0/2267958020 shutdown_connections 2024-06-18T13:02:23.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:23.774+0000 7f0d1f947640 1 -- 172.21.15.167:0/2267958020 wait complete. 2024-06-18T13:02:23.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:23 smithi179 ceph-mon[33312]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-18T13:02:24.503 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi167 2024-06-18T13:02:24.503 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi167=foo.a' 2024-06-18T13:02:24.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2267958020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-18T13:02:24.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2267958020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-18T13:02:24.753 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:02:24.874 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2267958020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-18T13:02:25.729 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:25 smithi179 ceph-mon[33312]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-18T13:02:25.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:25 smithi167 ceph-mon[37701]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-18T13:02:25.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:25 smithi167 ceph-mon[29496]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-18T13:02:26.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.175+0000 7f42af9ec640 1 -- 172.21.15.179:0/2650323933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a8103020 msgr2=0x7f42a81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:26.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.175+0000 7f42af9ec640 1 --2- 172.21.15.179:0/2650323933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a8103020 0x7f42a81098b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009ca0 tx=0x7f42a002a8c0 comp rx=0 tx=0).stop 2024-06-18T13:02:26.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.175+0000 7f42af9ec640 1 -- 172.21.15.179:0/2650323933 shutdown_connections 2024-06-18T13:02:26.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.175+0000 7f42af9ec640 1 --2- 172.21.15.179:0/2650323933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a8103020 0x7f42a81098b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:26.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.175+0000 7f42af9ec640 1 --2- 172.21.15.179:0/2650323933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a8102660 0x7f42a8102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:26.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.175+0000 7f42af9ec640 1 --2- 172.21.15.179:0/2650323933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a80ff680 0x7f42a80ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:26.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.175+0000 7f42af9ec640 1 -- 172.21.15.179:0/2650323933 >> 172.21.15.179:0/2650323933 conn(0x7f42a80fb1f0 msgr2=0x7f42a80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:26.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.176+0000 7f42af9ec640 1 -- 172.21.15.179:0/2650323933 shutdown_connections 2024-06-18T13:02:26.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.176+0000 7f42af9ec640 1 -- 172.21.15.179:0/2650323933 wait complete. 2024-06-18T13:02:26.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.176+0000 7f42af9ec640 1 Processor -- start 2024-06-18T13:02:26.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.176+0000 7f42af9ec640 1 -- start start 2024-06-18T13:02:26.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.176+0000 7f42af9ec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a80ff680 0x7f42a8196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:26.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.176+0000 7f42af9ec640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a8102660 0x7f42a81971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:26.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.176+0000 7f42af9ec640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a8103020 0x7f42a819e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:26.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.176+0000 7f42af9ec640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42a810c550 con 0x7f42a80ff680 2024-06-18T13:02:26.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.176+0000 7f42af9ec640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f42a810c3d0 con 0x7f42a8102660 2024-06-18T13:02:26.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.176+0000 7f42af9ec640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f42a810c6d0 con 0x7f42a8103020 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.176+0000 7f42acf87640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a8102660 0x7f42a81971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.176+0000 7f42acf87640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a8102660 0x7f42a81971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60258/0 (socket says 172.21.15.179:60258) 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.176+0000 7f42acf87640 1 -- 172.21.15.179:0/2204253247 learned_addr learned my addr 172.21.15.179:0/2204253247 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.176+0000 7f42ad788640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a80ff680 0x7f42a8196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.177+0000 7f42adf89640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a8103020 0x7f42a819e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.177+0000 7f42acf87640 1 -- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a8103020 msgr2=0x7f42a819e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.177+0000 7f42acf87640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a8103020 0x7f42a819e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.177+0000 7f42acf87640 1 -- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a80ff680 msgr2=0x7f42a8196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.177+0000 7f42acf87640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a80ff680 0x7f42a8196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.177+0000 7f42acf87640 1 -- 172.21.15.179:0/2204253247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a819e8a0 con 0x7f42a8102660 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.177+0000 7f42ad788640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a80ff680 0x7f42a8196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.177+0000 7f42acf87640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a8102660 0x7f42a81971c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f429800d950 tx=0x7f429800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.177+0000 7f429e7fc640 1 -- 172.21.15.179:0/2204253247 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298017030 con 0x7f42a8102660 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.177+0000 7f429e7fc640 1 -- 172.21.15.179:0/2204253247 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f4298004020 con 0x7f42a8102660 2024-06-18T13:02:26.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.177+0000 7f42af9ec640 1 -- 172.21.15.179:0/2204253247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a819eb90 con 0x7f42a8102660 2024-06-18T13:02:26.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.177+0000 7f429e7fc640 1 -- 172.21.15.179:0/2204253247 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298005570 con 0x7f42a8102660 2024-06-18T13:02:26.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.177+0000 7f42af9ec640 1 -- 172.21.15.179:0/2204253247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a819f120 con 0x7f42a8102660 2024-06-18T13:02:26.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.178+0000 7f42af9ec640 1 -- 172.21.15.179:0/2204253247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a80ffa80 con 0x7f42a8102660 2024-06-18T13:02:26.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.179+0000 7f429e7fc640 1 -- 172.21.15.179:0/2204253247 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) ==== 92117+0+0 (secure 0 0 0) 0x7f4298009a30 con 0x7f42a8102660 2024-06-18T13:02:26.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.179+0000 7f429e7fc640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f429406df50 0x7f4294070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:26.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.179+0000 7f429e7fc640 1 -- 172.21.15.179:0/2204253247 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5287+0+0 (secure 0 0 0) 0x7f429808d6a0 con 0x7f42a8102660 2024-06-18T13:02:26.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.179+0000 7f42ad788640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f429406df50 0x7f4294070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:26.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.180+0000 7f42ad788640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f429406df50 0x7f4294070410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f42a4005fd0 tx=0x7f42a4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:26.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.184+0000 7f429e7fc640 1 -- 172.21.15.179:0/2204253247 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f429805b370 con 0x7f42a8102660 2024-06-18T13:02:26.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.370+0000 7f42af9ec640 1 -- 172.21.15.179:0/2204253247 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi167=foo.a", "target": ["mon-mgr", ""]}) -- 0x7f42a8061de0 con 0x7f429406df50 2024-06-18T13:02:26.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.376+0000 7f429e7fc640 1 -- 172.21.15.179:0/2204253247 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f42a8061de0 con 0x7f429406df50 2024-06-18T13:02:26.376 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled rgw.foo.a update... 2024-06-18T13:02:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.379+0000 7f42af9ec640 1 -- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f429406df50 msgr2=0x7f4294070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.379+0000 7f42af9ec640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f429406df50 0x7f4294070410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f42a4005fd0 tx=0x7f42a4005850 comp rx=0 tx=0).stop 2024-06-18T13:02:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.379+0000 7f42af9ec640 1 -- 172.21.15.179:0/2204253247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a8102660 msgr2=0x7f42a81971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.379+0000 7f42af9ec640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a8102660 0x7f42a81971c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f429800d950 tx=0x7f429800de10 comp rx=0 tx=0).stop 2024-06-18T13:02:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.379+0000 7f42af9ec640 1 -- 172.21.15.179:0/2204253247 shutdown_connections 2024-06-18T13:02:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.379+0000 7f42af9ec640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f429406df50 0x7f4294070410 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.379+0000 7f42af9ec640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a8103020 0x7f42a819e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.379+0000 7f42af9ec640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a8102660 0x7f42a81971c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.379+0000 7f42af9ec640 1 --2- 172.21.15.179:0/2204253247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a80ff680 0x7f42a8196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.379+0000 7f42af9ec640 1 -- 172.21.15.179:0/2204253247 >> 172.21.15.179:0/2204253247 conn(0x7f42a80fb1f0 msgr2=0x7f42a80fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.379+0000 7f42af9ec640 1 -- 172.21.15.179:0/2204253247 shutdown_connections 2024-06-18T13:02:26.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:26.379+0000 7f42af9ec640 1 -- 172.21.15.179:0/2204253247 wait complete. 2024-06-18T13:02:26.971 DEBUG:teuthology.orchestra.run.smithi167:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@rgw.foo.a.service 2024-06-18T13:02:26.974 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi179 2024-06-18T13:02:26.974 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-06-18T13:02:27.210 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:02:27.597 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:27 smithi179 ceph-mon[33312]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi167=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:27.598 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:27 smithi179 ceph-mon[33312]: Saving service rgw.foo.a spec with placement smithi167=foo.a;count:1 2024-06-18T13:02:27.598 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:27 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:27.598 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:27 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:27.598 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:27 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:27.598 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:27 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:27.598 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:27 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:27.598 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:27 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-18T13:02:27.598 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:27 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-18T13:02:27.598 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:27 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:27.598 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:27 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:27.598 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:27 smithi179 ceph-mon[33312]: Deploying daemon rgw.foo.a on smithi167 2024-06-18T13:02:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[29496]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi167=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[29496]: Saving service rgw.foo.a spec with placement smithi167=foo.a;count:1 2024-06-18T13:02:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:27.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:27.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:27.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:27.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:27.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-18T13:02:27.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-18T13:02:27.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:27.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:27.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[29496]: Deploying daemon rgw.foo.a on smithi167 2024-06-18T13:02:27.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[37701]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi167=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:27.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[37701]: Saving service rgw.foo.a spec with placement smithi167=foo.a;count:1 2024-06-18T13:02:27.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:27.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:27.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:27.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:27.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:27.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-18T13:02:27.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-18T13:02:27.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:27.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:27.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:27 smithi167 ceph-mon[37701]: Deploying daemon rgw.foo.a on smithi167 2024-06-18T13:02:28.383 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:02:28 smithi167 systemd[1]: Starting Ceph rgw.foo.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:02:28.596 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:28 smithi179 ceph-mon[33312]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:02:28.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.681+0000 7f269b34b640 1 -- 172.21.15.179:0/1955197552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26941013b0 msgr2=0x7f26941017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:28.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.681+0000 7f269b34b640 1 --2- 172.21.15.179:0/1955197552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26941013b0 0x7f26941017b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f268c009900 tx=0x7f268c02a410 comp rx=0 tx=0).stop 2024-06-18T13:02:28.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.681+0000 7f269b34b640 1 -- 172.21.15.179:0/1955197552 shutdown_connections 2024-06-18T13:02:28.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.681+0000 7f269b34b640 1 --2- 172.21.15.179:0/1955197552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2694102f70 0x7f2694109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:28.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.681+0000 7f269b34b640 1 --2- 172.21.15.179:0/1955197552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26941025b0 0x7f2694102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:28.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.681+0000 7f269b34b640 1 --2- 172.21.15.179:0/1955197552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26941013b0 0x7f26941017b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:28.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.681+0000 7f269b34b640 1 -- 172.21.15.179:0/1955197552 >> 172.21.15.179:0/1955197552 conn(0x7f26940fcb40 msgr2=0x7f26940fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:28.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.681+0000 7f269b34b640 1 -- 172.21.15.179:0/1955197552 shutdown_connections 2024-06-18T13:02:28.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.681+0000 7f269b34b640 1 -- 172.21.15.179:0/1955197552 wait complete. 2024-06-18T13:02:28.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.682+0000 7f269b34b640 1 Processor -- start 2024-06-18T13:02:28.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.682+0000 7f269b34b640 1 -- start start 2024-06-18T13:02:28.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.682+0000 7f269b34b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26941013b0 0x7f2694196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:28.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.682+0000 7f269b34b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26941025b0 0x7f2694197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:28.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.682+0000 7f269b34b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2694102f70 0x7f269419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:28.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.682+0000 7f269b34b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f269410c520 con 0x7f2694102f70 2024-06-18T13:02:28.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.682+0000 7f269b34b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f269410c3a0 con 0x7f26941025b0 2024-06-18T13:02:28.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.682+0000 7f269b34b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f269410c6a0 con 0x7f26941013b0 2024-06-18T13:02:28.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.682+0000 7f26988e6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26941025b0 0x7f2694197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:28.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.682+0000 7f26988e6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26941025b0 0x7f2694197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60280/0 (socket says 172.21.15.179:60280) 2024-06-18T13:02:28.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.682+0000 7f26988e6640 1 -- 172.21.15.179:0/1551964079 learned_addr learned my addr 172.21.15.179:0/1551964079 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T13:02:28.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.683+0000 7f26998e8640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2694102f70 0x7f269419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:28.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.683+0000 7f26990e7640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26941013b0 0x7f2694196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:28.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.683+0000 7f26988e6640 1 -- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26941013b0 msgr2=0x7f2694196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:28.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.683+0000 7f26988e6640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26941013b0 0x7f2694196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:28.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.683+0000 7f26988e6640 1 -- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2694102f70 msgr2=0x7f269419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:28.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.683+0000 7f26988e6640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2694102f70 0x7f269419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:28.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.683+0000 7f26988e6640 1 -- 172.21.15.179:0/1551964079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269419e840 con 0x7f26941025b0 2024-06-18T13:02:28.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.683+0000 7f26998e8640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2694102f70 0x7f269419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:02:28.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.683+0000 7f26988e6640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26941025b0 0x7f2694197160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f268400d950 tx=0x7f268400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:28.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.683+0000 7f268a7fc640 1 -- 172.21.15.179:0/1551964079 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684017030 con 0x7f26941025b0 2024-06-18T13:02:28.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.683+0000 7f268a7fc640 1 -- 172.21.15.179:0/1551964079 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f2684004020 con 0x7f26941025b0 2024-06-18T13:02:28.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.683+0000 7f269b34b640 1 -- 172.21.15.179:0/1551964079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269419eb30 con 0x7f26941025b0 2024-06-18T13:02:28.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.683+0000 7f268a7fc640 1 -- 172.21.15.179:0/1551964079 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684005570 con 0x7f26941025b0 2024-06-18T13:02:28.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.683+0000 7f269b34b640 1 -- 172.21.15.179:0/1551964079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f269419f090 con 0x7f26941025b0 2024-06-18T13:02:28.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.684+0000 7f269b34b640 1 -- 172.21.15.179:0/1551964079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f269410cc80 con 0x7f26941025b0 2024-06-18T13:02:28.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.685+0000 7f268a7fc640 1 -- 172.21.15.179:0/1551964079 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) ==== 92117+0+0 (secure 0 0 0) 0x7f2684009a30 con 0x7f26941025b0 2024-06-18T13:02:28.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.685+0000 7f268a7fc640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f268006df50 0x7f2680070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:28.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.685+0000 7f268a7fc640 1 -- 172.21.15.179:0/1551964079 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5287+0+0 (secure 0 0 0) 0x7f268408d5f0 con 0x7f26941025b0 2024-06-18T13:02:28.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.685+0000 7f26990e7640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f268006df50 0x7f2680070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:28.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.686+0000 7f26990e7640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f268006df50 0x7f2680070410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f268c0057c0 tx=0x7f268c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:28.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.690+0000 7f268a7fc640 1 -- 172.21.15.179:0/1551964079 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f268405ac70 con 0x7f26941025b0 2024-06-18T13:02:28.708 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:28 smithi167 ceph-mon[29496]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:02:28.709 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:28 smithi167 ceph-mon[37701]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:02:28.709 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:02:28 smithi167 podman[63614]: 2024-06-18 13:02:28.506927275 +0000 UTC m=+0.107829495 container create 726db2c5d89f783ca7488a694be2443acc6c76107b42e396e9b5f0c5c84a5eea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-rgw-foo-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-3781e3a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612) 2024-06-18T13:02:28.709 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:02:28 smithi167 podman[63614]: 2024-06-18 13:02:28.41620217 +0000 UTC m=+0.017104400 image pull af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314 2024-06-18T13:02:28.709 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:02:28 smithi167 podman[63614]: 2024-06-18 13:02:28.709824163 +0000 UTC m=+0.310726393 container init 726db2c5d89f783ca7488a694be2443acc6c76107b42e396e9b5f0c5c84a5eea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-rgw-foo-a, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-3781e3a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240612, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-18T13:02:28.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:28.864+0000 7f269b34b640 1 -- 172.21.15.179:0/1551964079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) -- 0x7f269402d5a0 con 0x7f26941025b0 2024-06-18T13:02:28.976 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:02:28 smithi167 podman[63614]: 2024-06-18 13:02:28.712828749 +0000 UTC m=+0.313730974 container start 726db2c5d89f783ca7488a694be2443acc6c76107b42e396e9b5f0c5c84a5eea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-rgw-foo-a, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-3781e3a, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-06-18T13:02:28.976 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:02:28 smithi167 bash[63614]: 726db2c5d89f783ca7488a694be2443acc6c76107b42e396e9b5f0c5c84a5eea 2024-06-18T13:02:28.976 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:02:28 smithi167 systemd[1]: Started Ceph rgw.foo.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:02:29.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:29.394+0000 7f268a7fc640 1 -- 172.21.15.179:0/1551964079 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v61) ==== 160+0+0 (secure 0 0 0) 0x7f268405ef70 con 0x7f26941025b0 2024-06-18T13:02:29.394 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'datapool' created 2024-06-18T13:02:29.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:29.398+0000 7f269b34b640 1 -- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f268006df50 msgr2=0x7f2680070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:29.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:29.398+0000 7f269b34b640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f268006df50 0x7f2680070410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f268c0057c0 tx=0x7f268c0058d0 comp rx=0 tx=0).stop 2024-06-18T13:02:29.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:29.398+0000 7f269b34b640 1 -- 172.21.15.179:0/1551964079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26941025b0 msgr2=0x7f2694197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:29.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:29.398+0000 7f269b34b640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26941025b0 0x7f2694197160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f268400d950 tx=0x7f268400de10 comp rx=0 tx=0).stop 2024-06-18T13:02:29.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:29.398+0000 7f269b34b640 1 -- 172.21.15.179:0/1551964079 shutdown_connections 2024-06-18T13:02:29.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:29.398+0000 7f269b34b640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f268006df50 0x7f2680070410 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:29.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:29.398+0000 7f269b34b640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2694102f70 0x7f269419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:29.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:29.398+0000 7f269b34b640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26941025b0 0x7f2694197160 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:29.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:29.398+0000 7f269b34b640 1 --2- 172.21.15.179:0/1551964079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26941013b0 0x7f2694196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:29.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:29.398+0000 7f269b34b640 1 -- 172.21.15.179:0/1551964079 >> 172.21.15.179:0/1551964079 conn(0x7f26940fcb40 msgr2=0x7f26941059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:29.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:29.399+0000 7f269b34b640 1 -- 172.21.15.179:0/1551964079 shutdown_connections 2024-06-18T13:02:29.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:29.399+0000 7f269b34b640 1 -- 172.21.15.179:0/1551964079 wait complete. 2024-06-18T13:02:29.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.179:0/1551964079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-06-18T13:02:29.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-06-18T13:02:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:29 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:29 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:29 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:29 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:29 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:29 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:29.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.179:0/1551964079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-06-18T13:02:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-06-18T13:02:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.179:0/1551964079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-06-18T13:02:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-06-18T13:02:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:29.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:29 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:30.019 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- rbd pool init datapool 2024-06-18T13:02:30.256 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:02:30.631 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:30 smithi179 ceph-mon[33312]: Saving service rgw.foo.a spec with placement smithi167=foo.a;count:1 2024-06-18T13:02:30.632 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-06-18T13:02:30.632 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:30 smithi179 ceph-mon[33312]: osdmap e61: 8 total, 8 up, 8 in 2024-06-18T13:02:30.632 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3464753863' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-18T13:02:30.632 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:30 smithi179 ceph-mon[33312]: pgmap v179: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:02:30.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:30 smithi167 ceph-mon[29496]: Saving service rgw.foo.a spec with placement smithi167=foo.a;count:1 2024-06-18T13:02:30.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-06-18T13:02:30.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:30 smithi167 ceph-mon[29496]: osdmap e61: 8 total, 8 up, 8 in 2024-06-18T13:02:30.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3464753863' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-18T13:02:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:30 smithi167 ceph-mon[29496]: pgmap v179: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:02:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:30 smithi167 ceph-mon[37701]: Saving service rgw.foo.a spec with placement smithi167=foo.a;count:1 2024-06-18T13:02:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-06-18T13:02:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:30 smithi167 ceph-mon[37701]: osdmap e61: 8 total, 8 up, 8 in 2024-06-18T13:02:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3464753863' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-18T13:02:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:30 smithi167 ceph-mon[37701]: pgmap v179: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-18T13:02:31.424 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:31 smithi179 ceph-mon[33312]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:02:31.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3464753863' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-06-18T13:02:31.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:31 smithi179 ceph-mon[33312]: osdmap e62: 8 total, 8 up, 8 in 2024-06-18T13:02:31.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:31 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:31.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:31 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:31.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:31 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:31.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:31 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:31.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:31 smithi167 ceph-mon[37701]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:02:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3464753863' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-06-18T13:02:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:31 smithi167 ceph-mon[37701]: osdmap e62: 8 total, 8 up, 8 in 2024-06-18T13:02:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:31 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:31 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:31 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:31 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:31 smithi167 ceph-mon[29496]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:02:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3464753863' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-06-18T13:02:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:31 smithi167 ceph-mon[29496]: osdmap e62: 8 total, 8 up, 8 in 2024-06-18T13:02:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:31 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:31 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:31 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:31.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:31 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:32.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[37701]: Checking dashboard <-> RGW credentials 2024-06-18T13:02:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[37701]: osdmap e63: 8 total, 8 up, 8 in 2024-06-18T13:02:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/806756546' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-18T13:02:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3533999533' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-18T13:02:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-18T13:02:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[37701]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-18T13:02:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[37701]: pgmap v182: 68 pgs: 10 creating+peering, 6 active+clean, 52 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 341 B/s wr, 0 op/s 2024-06-18T13:02:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.179:0/2753584082' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-06-18T13:02:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-06-18T13:02:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-06-18T13:02:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[37701]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-06-18T13:02:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-06-18T13:02:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[37701]: osdmap e64: 8 total, 8 up, 8 in 2024-06-18T13:02:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[29496]: Checking dashboard <-> RGW credentials 2024-06-18T13:02:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[29496]: osdmap e63: 8 total, 8 up, 8 in 2024-06-18T13:02:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/806756546' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-18T13:02:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3533999533' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-18T13:02:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-18T13:02:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[29496]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-18T13:02:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[29496]: pgmap v182: 68 pgs: 10 creating+peering, 6 active+clean, 52 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 341 B/s wr, 0 op/s 2024-06-18T13:02:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.179:0/2753584082' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-06-18T13:02:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-06-18T13:02:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-06-18T13:02:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[29496]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-06-18T13:02:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-06-18T13:02:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:32 smithi167 ceph-mon[29496]: osdmap e64: 8 total, 8 up, 8 in 2024-06-18T13:02:32.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:32 smithi179 ceph-mon[33312]: Checking dashboard <-> RGW credentials 2024-06-18T13:02:32.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:32 smithi179 ceph-mon[33312]: osdmap e63: 8 total, 8 up, 8 in 2024-06-18T13:02:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/806756546' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-18T13:02:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3533999533' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-18T13:02:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-18T13:02:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:32 smithi179 ceph-mon[33312]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-18T13:02:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:32 smithi179 ceph-mon[33312]: pgmap v182: 68 pgs: 10 creating+peering, 6 active+clean, 52 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 341 B/s wr, 0 op/s 2024-06-18T13:02:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.179:0/2753584082' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-06-18T13:02:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-06-18T13:02:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-06-18T13:02:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:32 smithi179 ceph-mon[33312]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-06-18T13:02:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-06-18T13:02:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:32 smithi179 ceph-mon[33312]: osdmap e64: 8 total, 8 up, 8 in 2024-06-18T13:02:34.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:34 smithi167 ceph-mon[29496]: osdmap e65: 8 total, 8 up, 8 in 2024-06-18T13:02:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/806756546' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-18T13:02:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-18T13:02:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3533999533' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-18T13:02:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:34 smithi167 ceph-mon[29496]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-18T13:02:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:34 smithi167 ceph-mon[29496]: pgmap v185: 100 pgs: 18 creating+peering, 18 active+clean, 64 unknown; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-06-18T13:02:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:34 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:34 smithi167 ceph-mon[37701]: osdmap e65: 8 total, 8 up, 8 in 2024-06-18T13:02:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/806756546' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-18T13:02:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-18T13:02:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3533999533' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-18T13:02:34.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:34 smithi167 ceph-mon[37701]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-18T13:02:34.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:34 smithi167 ceph-mon[37701]: pgmap v185: 100 pgs: 18 creating+peering, 18 active+clean, 64 unknown; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-06-18T13:02:34.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:34 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:34.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:34 smithi179 ceph-mon[33312]: osdmap e65: 8 total, 8 up, 8 in 2024-06-18T13:02:34.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/806756546' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-18T13:02:34.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-18T13:02:34.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3533999533' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-18T13:02:34.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:34 smithi179 ceph-mon[33312]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-18T13:02:34.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:34 smithi179 ceph-mon[33312]: pgmap v185: 100 pgs: 18 creating+peering, 18 active+clean, 64 unknown; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-06-18T13:02:34.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:34 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:35.106 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.179 --placement '1;smithi179=iscsi.a' 2024-06-18T13:02:35.349 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:02:35.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-06-18T13:02:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:35 smithi167 ceph-mon[29496]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-06-18T13:02:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:35 smithi167 ceph-mon[29496]: osdmap e66: 8 total, 8 up, 8 in 2024-06-18T13:02:35.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-06-18T13:02:35.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:35 smithi167 ceph-mon[37701]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-06-18T13:02:35.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:35 smithi167 ceph-mon[37701]: osdmap e66: 8 total, 8 up, 8 in 2024-06-18T13:02:35.768 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-06-18T13:02:35.768 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:35 smithi179 ceph-mon[33312]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-06-18T13:02:35.768 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:35 smithi179 ceph-mon[33312]: osdmap e66: 8 total, 8 up, 8 in 2024-06-18T13:02:36.578 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:36 smithi179 ceph-mon[33312]: osdmap e67: 8 total, 8 up, 8 in 2024-06-18T13:02:36.578 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3533999533' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-18T13:02:36.578 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:36 smithi179 ceph-mon[33312]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-18T13:02:36.578 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/806756546' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-18T13:02:36.578 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-18T13:02:36.578 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:36 smithi179 ceph-mon[33312]: pgmap v188: 132 pgs: 16 creating+peering, 58 active+clean, 58 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:02:36.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:36 smithi167 ceph-mon[29496]: osdmap e67: 8 total, 8 up, 8 in 2024-06-18T13:02:36.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3533999533' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-18T13:02:36.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:36 smithi167 ceph-mon[29496]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-18T13:02:36.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/806756546' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-18T13:02:36.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-18T13:02:36.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:36 smithi167 ceph-mon[29496]: pgmap v188: 132 pgs: 16 creating+peering, 58 active+clean, 58 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:02:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:36 smithi167 ceph-mon[37701]: osdmap e67: 8 total, 8 up, 8 in 2024-06-18T13:02:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3533999533' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-18T13:02:36.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:36 smithi167 ceph-mon[37701]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-18T13:02:36.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/806756546' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-18T13:02:36.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-18T13:02:36.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:36 smithi167 ceph-mon[37701]: pgmap v188: 132 pgs: 16 creating+peering, 58 active+clean, 58 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:02:36.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.771+0000 7f10fa831640 1 -- 172.21.15.179:0/1555356792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10f4100280 msgr2=0x7f10f4100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:36.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.771+0000 7f10fa831640 1 --2- 172.21.15.179:0/1555356792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10f4100280 0x7f10f4100680 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009900 tx=0x7f10e402a410 comp rx=0 tx=0).stop 2024-06-18T13:02:36.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.771+0000 7f10fa831640 1 -- 172.21.15.179:0/1555356792 shutdown_connections 2024-06-18T13:02:36.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.771+0000 7f10fa831640 1 --2- 172.21.15.179:0/1555356792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f4101e40 0x7f10f41086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:36.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.771+0000 7f10fa831640 1 --2- 172.21.15.179:0/1555356792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f4101480 0x7f10f4101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:36.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.771+0000 7f10fa831640 1 --2- 172.21.15.179:0/1555356792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10f4100280 0x7f10f4100680 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:36.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.771+0000 7f10fa831640 1 -- 172.21.15.179:0/1555356792 >> 172.21.15.179:0/1555356792 conn(0x7f10f40fba30 msgr2=0x7f10f40fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:36.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.771+0000 7f10fa831640 1 -- 172.21.15.179:0/1555356792 shutdown_connections 2024-06-18T13:02:36.775 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.771+0000 7f10fa831640 1 -- 172.21.15.179:0/1555356792 wait complete. 2024-06-18T13:02:36.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10fa831640 1 Processor -- start 2024-06-18T13:02:36.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10fa831640 1 -- start start 2024-06-18T13:02:36.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10fa831640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10f4100280 0x7f10f410ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:36.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10fa831640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f4101480 0x7f10f4109250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:36.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10fa831640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f4101e40 0x7f10f4109790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:36.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10fa831640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10f410c4f0 con 0x7f10f4101e40 2024-06-18T13:02:36.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10fa831640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f10f410c370 con 0x7f10f4100280 2024-06-18T13:02:36.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10fa831640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f10f410c670 con 0x7f10f4101480 2024-06-18T13:02:36.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10f3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10f4100280 0x7f10f410ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:36.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10f3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10f4100280 0x7f10f410ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53580/0 (socket says 172.21.15.179:53580) 2024-06-18T13:02:36.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10f3fff640 1 -- 172.21.15.179:0/888545730 learned_addr learned my addr 172.21.15.179:0/888545730 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T13:02:36.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10ebfff640 1 --2- 172.21.15.179:0/888545730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f4101480 0x7f10f4109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:36.776 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10f8dce640 1 --2- 172.21.15.179:0/888545730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f4101e40 0x7f10f4109790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:36.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10f3fff640 1 -- 172.21.15.179:0/888545730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f4101480 msgr2=0x7f10f4109250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:36.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10f3fff640 1 --2- 172.21.15.179:0/888545730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f4101480 0x7f10f4109250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:36.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10f3fff640 1 -- 172.21.15.179:0/888545730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f4101e40 msgr2=0x7f10f4109790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:36.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10f3fff640 1 --2- 172.21.15.179:0/888545730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f4101e40 0x7f10f4109790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:36.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.772+0000 7f10f3fff640 1 -- 172.21.15.179:0/888545730 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f410a050 con 0x7f10f4100280 2024-06-18T13:02:36.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.773+0000 7f10f3fff640 1 --2- 172.21.15.179:0/888545730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10f4100280 0x7f10f410ac00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f10e4005940 tx=0x7f10e40045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:36.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.773+0000 7f10f1ffb640 1 -- 172.21.15.179:0/888545730 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e403f070 con 0x7f10f4100280 2024-06-18T13:02:36.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.773+0000 7f10fa831640 1 -- 172.21.15.179:0/888545730 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f41a0d70 con 0x7f10f4100280 2024-06-18T13:02:36.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.773+0000 7f10f1ffb640 1 -- 172.21.15.179:0/888545730 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f10e4005490 con 0x7f10f4100280 2024-06-18T13:02:36.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.773+0000 7f10f1ffb640 1 -- 172.21.15.179:0/888545730 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4035030 con 0x7f10f4100280 2024-06-18T13:02:36.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.773+0000 7f10fa831640 1 -- 172.21.15.179:0/888545730 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f41a1170 con 0x7f10f4100280 2024-06-18T13:02:36.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.774+0000 7f10fa831640 1 -- 172.21.15.179:0/888545730 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f4100680 con 0x7f10f4100280 2024-06-18T13:02:36.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.774+0000 7f10f1ffb640 1 -- 172.21.15.179:0/888545730 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) ==== 92117+0+0 (secure 0 0 0) 0x7f10e4042070 con 0x7f10f4100280 2024-06-18T13:02:36.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.775+0000 7f10f1ffb640 1 --2- 172.21.15.179:0/888545730 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f10e006df50 0x7f10e0070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:36.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.775+0000 7f10f1ffb640 1 -- 172.21.15.179:0/888545730 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7132+0+0 (secure 0 0 0) 0x7f10e40aeb70 con 0x7f10f4100280 2024-06-18T13:02:36.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.775+0000 7f10ebfff640 1 --2- 172.21.15.179:0/888545730 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f10e006df50 0x7f10e0070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:36.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.776+0000 7f10ebfff640 1 --2- 172.21.15.179:0/888545730 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f10e006df50 0x7f10e0070410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f10f410a980 tx=0x7f10dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:36.779 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.780+0000 7f10f1ffb640 1 -- 172.21.15.179:0/888545730 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f10e407ba40 con 0x7f10f4100280 2024-06-18T13:02:36.965 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.965+0000 7f10fa831640 1 -- 172.21.15.179:0/888545730 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.179", "placement": "1;smithi179=iscsi.a", "target": ["mon-mgr", ""]}) -- 0x7f10f4061eb0 con 0x7f10e006df50 2024-06-18T13:02:36.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.973+0000 7f10f1ffb640 1 -- 172.21.15.179:0/888545730 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+35 (secure 0 0 0) 0x7f10f4061eb0 con 0x7f10e006df50 2024-06-18T13:02:36.973 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled iscsi.datapool update... 2024-06-18T13:02:36.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.976+0000 7f10fa831640 1 -- 172.21.15.179:0/888545730 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f10e006df50 msgr2=0x7f10e0070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:36.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.976+0000 7f10fa831640 1 --2- 172.21.15.179:0/888545730 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f10e006df50 0x7f10e0070410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f10f410a980 tx=0x7f10dc005f90 comp rx=0 tx=0).stop 2024-06-18T13:02:36.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.976+0000 7f10fa831640 1 -- 172.21.15.179:0/888545730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10f4100280 msgr2=0x7f10f410ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:36.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.976+0000 7f10fa831640 1 --2- 172.21.15.179:0/888545730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10f4100280 0x7f10f410ac00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f10e4005940 tx=0x7f10e40045c0 comp rx=0 tx=0).stop 2024-06-18T13:02:36.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.976+0000 7f10fa831640 1 -- 172.21.15.179:0/888545730 shutdown_connections 2024-06-18T13:02:36.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.976+0000 7f10fa831640 1 --2- 172.21.15.179:0/888545730 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f10e006df50 0x7f10e0070410 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:36.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.976+0000 7f10fa831640 1 --2- 172.21.15.179:0/888545730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f4101e40 0x7f10f4109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:36.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.976+0000 7f10fa831640 1 --2- 172.21.15.179:0/888545730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f4101480 0x7f10f4109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:36.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.976+0000 7f10fa831640 1 --2- 172.21.15.179:0/888545730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10f4100280 0x7f10f410ac00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:36.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.976+0000 7f10fa831640 1 -- 172.21.15.179:0/888545730 >> 172.21.15.179:0/888545730 conn(0x7f10f40fba30 msgr2=0x7f10f40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:36.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.976+0000 7f10fa831640 1 -- 172.21.15.179:0/888545730 shutdown_connections 2024-06-18T13:02:36.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:36.976+0000 7f10fa831640 1 -- 172.21.15.179:0/888545730 wait complete. 2024-06-18T13:02:37.562 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:37 smithi179 ceph-mon[33312]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-06-18T13:02:37.562 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-06-18T13:02:37.562 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:37 smithi179 ceph-mon[33312]: osdmap e68: 8 total, 8 up, 8 in 2024-06-18T13:02:37.562 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/806756546' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-18T13:02:37.562 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3533999533' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-18T13:02:37.562 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-18T13:02:37.562 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:37 smithi179 ceph-mon[33312]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-18T13:02:37.562 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:37 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:37.596 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-06-18T13:02:37.596 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T13:02:37.596 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-06-18T13:02:37.632 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-18T13:02:37.632 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-06-18T13:02:37.660 DEBUG:teuthology.orchestra.run.smithi179:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@iscsi.iscsi.a.service 2024-06-18T13:02:37.703 INFO:tasks.cephadm:Adding prometheus.a on smithi179 2024-06-18T13:02:37.703 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch apply prometheus '1;smithi179=a' 2024-06-18T13:02:37.726 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-rgw-foo-a[63629]: 2024-06-18T13:02:37.501+0000 7f29ddce7880 -1 LDAP not started since no server URIs were provided in the configuration. 2024-06-18T13:02:37.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[29496]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-06-18T13:02:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-06-18T13:02:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[29496]: osdmap e68: 8 total, 8 up, 8 in 2024-06-18T13:02:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/806756546' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-18T13:02:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3533999533' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-18T13:02:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-18T13:02:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[29496]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-18T13:02:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[37701]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-06-18T13:02:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-06-18T13:02:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[37701]: osdmap e68: 8 total, 8 up, 8 in 2024-06-18T13:02:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/806756546' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-18T13:02:37.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3533999533' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-18T13:02:37.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-18T13:02:37.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[37701]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-18T13:02:37.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:37 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:37.993 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:02:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[37701]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.179", "placement": "1;smithi179=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[37701]: Saving service iscsi.datapool spec with placement smithi179=iscsi.a;count:1 2024-06-18T13:02:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-18T13:02:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[37701]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-18T13:02:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[37701]: osdmap e69: 8 total, 8 up, 8 in 2024-06-18T13:02:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[37701]: pgmap v191: 132 pgs: 15 creating+peering, 95 active+clean, 22 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-06-18T13:02:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-18T13:02:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-06-18T13:02:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[29496]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.179", "placement": "1;smithi179=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[29496]: Saving service iscsi.datapool spec with placement smithi179=iscsi.a;count:1 2024-06-18T13:02:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-18T13:02:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[29496]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-18T13:02:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[29496]: osdmap e69: 8 total, 8 up, 8 in 2024-06-18T13:02:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[29496]: pgmap v191: 132 pgs: 15 creating+peering, 95 active+clean, 22 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-06-18T13:02:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-18T13:02:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-06-18T13:02:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:38 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:38.810 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:38 smithi179 ceph-mon[33312]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.179", "placement": "1;smithi179=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:38.810 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:38 smithi179 ceph-mon[33312]: Saving service iscsi.datapool spec with placement smithi179=iscsi.a;count:1 2024-06-18T13:02:38.810 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-18T13:02:38.810 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:38 smithi179 ceph-mon[33312]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-18T13:02:38.810 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:38 smithi179 ceph-mon[33312]: osdmap e69: 8 total, 8 up, 8 in 2024-06-18T13:02:38.810 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:38 smithi179 ceph-mon[33312]: pgmap v191: 132 pgs: 15 creating+peering, 95 active+clean, 22 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-06-18T13:02:38.810 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:38 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:38.810 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:38 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:38.810 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:38 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:38.811 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:38 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-18T13:02:38.811 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:38 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-06-18T13:02:38.811 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:38 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:39.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:39 smithi167 ceph-mon[37701]: Deploying daemon iscsi.iscsi.a on smithi179 2024-06-18T13:02:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:39 smithi167 ceph-mon[37701]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-18T13:02:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:39 smithi167 ceph-mon[37701]: Cluster is now healthy 2024-06-18T13:02:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:39 smithi167 ceph-mon[29496]: Deploying daemon iscsi.iscsi.a on smithi179 2024-06-18T13:02:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:39 smithi167 ceph-mon[29496]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-18T13:02:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:39 smithi167 ceph-mon[29496]: Cluster is now healthy 2024-06-18T13:02:39.857 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:39 smithi179 ceph-mon[33312]: Deploying daemon iscsi.iscsi.a on smithi179 2024-06-18T13:02:39.857 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:39 smithi179 ceph-mon[33312]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-18T13:02:39.857 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:39 smithi179 ceph-mon[33312]: Cluster is now healthy 2024-06-18T13:02:40.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:40 smithi167 ceph-mon[37701]: pgmap v192: 132 pgs: 7 creating+peering, 112 active+clean, 13 unknown; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 2.3 KiB/s wr, 51 op/s 2024-06-18T13:02:40.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:40 smithi167 ceph-mon[29496]: pgmap v192: 132 pgs: 7 creating+peering, 112 active+clean, 13 unknown; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 2.3 KiB/s wr, 51 op/s 2024-06-18T13:02:40.732 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:40 smithi179 ceph-mon[33312]: pgmap v192: 132 pgs: 7 creating+peering, 112 active+clean, 13 unknown; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 2.3 KiB/s wr, 51 op/s 2024-06-18T13:02:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.816+0000 7f44d1b2a640 1 -- 172.21.15.179:0/2281411931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44cc1025b0 msgr2=0x7f44cc102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.816+0000 7f44d1b2a640 1 --2- 172.21.15.179:0/2281411931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44cc1025b0 0x7f44cc102a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009900 tx=0x7f44c002a410 comp rx=0 tx=0).stop 2024-06-18T13:02:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.816+0000 7f44d1b2a640 1 -- 172.21.15.179:0/2281411931 shutdown_connections 2024-06-18T13:02:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.816+0000 7f44d1b2a640 1 --2- 172.21.15.179:0/2281411931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44cc102f70 0x7f44cc109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.816+0000 7f44d1b2a640 1 --2- 172.21.15.179:0/2281411931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44cc1025b0 0x7f44cc102a30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.816+0000 7f44d1b2a640 1 --2- 172.21.15.179:0/2281411931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44cc1013b0 0x7f44cc1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.816+0000 7f44d1b2a640 1 -- 172.21.15.179:0/2281411931 >> 172.21.15.179:0/2281411931 conn(0x7f44cc0fcb40 msgr2=0x7f44cc0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.816+0000 7f44d1b2a640 1 -- 172.21.15.179:0/2281411931 shutdown_connections 2024-06-18T13:02:40.818 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.816+0000 7f44d1b2a640 1 -- 172.21.15.179:0/2281411931 wait complete. 2024-06-18T13:02:40.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.817+0000 7f44d1b2a640 1 Processor -- start 2024-06-18T13:02:40.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.817+0000 7f44d1b2a640 1 -- start start 2024-06-18T13:02:40.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44d1b2a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44cc1013b0 0x7f44cc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:40.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44d1b2a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44cc1025b0 0x7f44cc073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:40.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44d1b2a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44cc102f70 0x7f44cc0742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:40.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44d1b2a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc10c570 con 0x7f44cc1025b0 2024-06-18T13:02:40.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44d1b2a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc10c3f0 con 0x7f44cc1013b0 2024-06-18T13:02:40.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44d1b2a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f44cc10c6f0 con 0x7f44cc102f70 2024-06-18T13:02:40.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44cb7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44cc1013b0 0x7f44cc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:40.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44cb7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44cc1013b0 0x7f44cc075740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49434/0 (socket says 172.21.15.179:49434) 2024-06-18T13:02:40.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44cb7fe640 1 -- 172.21.15.179:0/1870864933 learned_addr learned my addr 172.21.15.179:0/1870864933 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T13:02:40.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44cbfff640 1 --2- 172.21.15.179:0/1870864933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44cc102f70 0x7f44cc0742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:40.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44caffd640 1 --2- 172.21.15.179:0/1870864933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44cc1025b0 0x7f44cc073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:40.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44cb7fe640 1 -- 172.21.15.179:0/1870864933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44cc102f70 msgr2=0x7f44cc0742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:40.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44cb7fe640 1 --2- 172.21.15.179:0/1870864933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44cc102f70 0x7f44cc0742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:40.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44cb7fe640 1 -- 172.21.15.179:0/1870864933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44cc1025b0 msgr2=0x7f44cc073d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:40.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44cb7fe640 1 --2- 172.21.15.179:0/1870864933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44cc1025b0 0x7f44cc073d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:40.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44cb7fe640 1 -- 172.21.15.179:0/1870864933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44cc074b90 con 0x7f44cc1013b0 2024-06-18T13:02:40.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44cb7fe640 1 --2- 172.21.15.179:0/1870864933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44cc1013b0 0x7f44cc075740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00d770 tx=0x7f44bc00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:40.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.818+0000 7f44c8ff9640 1 -- 172.21.15.179:0/1870864933 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc017070 con 0x7f44cc1013b0 2024-06-18T13:02:40.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.819+0000 7f44d1b2a640 1 -- 172.21.15.179:0/1870864933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44cc1a0d30 con 0x7f44cc1013b0 2024-06-18T13:02:40.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.819+0000 7f44c8ff9640 1 -- 172.21.15.179:0/1870864933 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f44bc009c30 con 0x7f44cc1013b0 2024-06-18T13:02:40.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.819+0000 7f44c8ff9640 1 -- 172.21.15.179:0/1870864933 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc003b80 con 0x7f44cc1013b0 2024-06-18T13:02:40.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.819+0000 7f44d1b2a640 1 -- 172.21.15.179:0/1870864933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44cc1a1180 con 0x7f44cc1013b0 2024-06-18T13:02:40.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.820+0000 7f44d1b2a640 1 -- 172.21.15.179:0/1870864933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44cc1017b0 con 0x7f44cc1013b0 2024-06-18T13:02:40.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.820+0000 7f44c8ff9640 1 -- 172.21.15.179:0/1870864933 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) ==== 92117+0+0 (secure 0 0 0) 0x7f44bc003d20 con 0x7f44cc1013b0 2024-06-18T13:02:40.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.821+0000 7f44c8ff9640 1 --2- 172.21.15.179:0/1870864933 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f44b406df50 0x7f44b4070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:40.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.821+0000 7f44c8ff9640 1 -- 172.21.15.179:0/1870864933 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7148+0+0 (secure 0 0 0) 0x7f44bc08dd30 con 0x7f44cc1013b0 2024-06-18T13:02:40.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.822+0000 7f44caffd640 1 --2- 172.21.15.179:0/1870864933 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f44b406df50 0x7f44b4070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:40.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.822+0000 7f44caffd640 1 --2- 172.21.15.179:0/1870864933 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f44b406df50 0x7f44b4070410 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f44c0006010 tx=0x7f44c0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:40.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:40.825+0000 7f44c8ff9640 1 -- 172.21.15.179:0/1870864933 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f44bc05ac70 con 0x7f44cc1013b0 2024-06-18T13:02:41.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:41.057+0000 7f44d1b2a640 1 -- 172.21.15.179:0/1870864933 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}) -- 0x7f44cc061de0 con 0x7f44b406df50 2024-06-18T13:02:41.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:41.064+0000 7f44c8ff9640 1 -- 172.21.15.179:0/1870864933 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+31 (secure 0 0 0) 0x7f44cc061de0 con 0x7f44b406df50 2024-06-18T13:02:41.065 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled prometheus update... 2024-06-18T13:02:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:41.067+0000 7f44d1b2a640 1 -- 172.21.15.179:0/1870864933 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f44b406df50 msgr2=0x7f44b4070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:41.067+0000 7f44d1b2a640 1 --2- 172.21.15.179:0/1870864933 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f44b406df50 0x7f44b4070410 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f44c0006010 tx=0x7f44c0000bc0 comp rx=0 tx=0).stop 2024-06-18T13:02:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:41.067+0000 7f44d1b2a640 1 -- 172.21.15.179:0/1870864933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44cc1013b0 msgr2=0x7f44cc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:41.067+0000 7f44d1b2a640 1 --2- 172.21.15.179:0/1870864933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44cc1013b0 0x7f44cc075740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00d770 tx=0x7f44bc00dc30 comp rx=0 tx=0).stop 2024-06-18T13:02:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:41.067+0000 7f44d1b2a640 1 -- 172.21.15.179:0/1870864933 shutdown_connections 2024-06-18T13:02:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:41.067+0000 7f44d1b2a640 1 --2- 172.21.15.179:0/1870864933 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f44b406df50 0x7f44b4070410 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:41.067+0000 7f44d1b2a640 1 --2- 172.21.15.179:0/1870864933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44cc102f70 0x7f44cc0742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:41.067+0000 7f44d1b2a640 1 --2- 172.21.15.179:0/1870864933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44cc1025b0 0x7f44cc073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:41.067+0000 7f44d1b2a640 1 --2- 172.21.15.179:0/1870864933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44cc1013b0 0x7f44cc075740 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:41.067+0000 7f44d1b2a640 1 -- 172.21.15.179:0/1870864933 >> 172.21.15.179:0/1870864933 conn(0x7f44cc0fcb40 msgr2=0x7f44cc1059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:41.067+0000 7f44d1b2a640 1 -- 172.21.15.179:0/1870864933 shutdown_connections 2024-06-18T13:02:41.068 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:41.067+0000 7f44d1b2a640 1 -- 172.21.15.179:0/1870864933 wait complete. 2024-06-18T13:02:41.942 DEBUG:teuthology.orchestra.run.smithi179:prometheus.a> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@prometheus.a.service 2024-06-18T13:02:41.945 INFO:tasks.cephadm:Adding node-exporter.a on smithi167 2024-06-18T13:02:41.945 INFO:tasks.cephadm:Adding node-exporter.b on smithi179 2024-06-18T13:02:41.945 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch apply node-exporter '2;smithi167=a;smithi179=b' 2024-06-18T13:02:42.328 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:42 smithi179 ceph-mon[33312]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:42.329 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:42 smithi179 ceph-mon[33312]: Saving service prometheus spec with placement smithi179=a;count:1 2024-06-18T13:02:42.329 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:42 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:42.329 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:42 smithi179 ceph-mon[33312]: pgmap v193: 132 pgs: 132 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 4.3 KiB/s wr, 131 op/s 2024-06-18T13:02:42.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:42 smithi167 ceph-mon[37701]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:42.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:42 smithi167 ceph-mon[37701]: Saving service prometheus spec with placement smithi179=a;count:1 2024-06-18T13:02:42.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:42 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:42.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:42 smithi167 ceph-mon[37701]: pgmap v193: 132 pgs: 132 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 4.3 KiB/s wr, 131 op/s 2024-06-18T13:02:42.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:42 smithi167 ceph-mon[29496]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:42.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:42 smithi167 ceph-mon[29496]: Saving service prometheus spec with placement smithi179=a;count:1 2024-06-18T13:02:42.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:42 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:42.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:42 smithi167 ceph-mon[29496]: pgmap v193: 132 pgs: 132 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 4.3 KiB/s wr, 131 op/s 2024-06-18T13:02:42.731 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:02:43.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:43 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:43.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:43 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:43.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:43 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:43.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:43 smithi167 ceph-mon[37701]: Checking pool "datapool" exists for service iscsi.datapool 2024-06-18T13:02:43.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:43 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:43.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:43 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:43.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.179:0/678471533' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-18T13:02:43.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:43 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:43.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:43 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:43.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:43 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:43.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:43 smithi167 ceph-mon[29496]: Checking pool "datapool" exists for service iscsi.datapool 2024-06-18T13:02:43.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:43 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:43.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:43 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:43.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.179:0/678471533' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-18T13:02:43.526 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:43 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:43.526 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:43 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:43.526 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:43 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:43.526 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:43 smithi179 ceph-mon[33312]: Checking pool "datapool" exists for service iscsi.datapool 2024-06-18T13:02:43.526 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:43 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:43.526 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:43 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:02:43.526 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.179:0/678471533' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-18T13:02:44.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.303+0000 7fc7162e5640 1 -- 172.21.15.179:0/3258867798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc710104740 msgr2=0x7fc710106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:44.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.303+0000 7fc7162e5640 1 --2- 172.21.15.179:0/3258867798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc710104740 0x7fc710106b30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009900 tx=0x7fc6f802a410 comp rx=0 tx=0).stop 2024-06-18T13:02:44.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.303+0000 7fc7162e5640 1 -- 172.21.15.179:0/3258867798 shutdown_connections 2024-06-18T13:02:44.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.303+0000 7fc7162e5640 1 --2- 172.21.15.179:0/3258867798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc710107070 0x7fc7101094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:44.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.303+0000 7fc7162e5640 1 --2- 172.21.15.179:0/3258867798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc710104740 0x7fc710106b30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:44.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.303+0000 7fc7162e5640 1 --2- 172.21.15.179:0/3258867798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc710101e10 0x7fc710104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:44.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.303+0000 7fc7162e5640 1 -- 172.21.15.179:0/3258867798 >> 172.21.15.179:0/3258867798 conn(0x7fc7100fba30 msgr2=0x7fc7100fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:44.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.303+0000 7fc7162e5640 1 -- 172.21.15.179:0/3258867798 shutdown_connections 2024-06-18T13:02:44.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.303+0000 7fc7162e5640 1 -- 172.21.15.179:0/3258867798 wait complete. 2024-06-18T13:02:44.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc7162e5640 1 Processor -- start 2024-06-18T13:02:44.306 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc7162e5640 1 -- start start 2024-06-18T13:02:44.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc7162e5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc710101e10 0x7fc710196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:44.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc7162e5640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc710104740 0x7fc7101970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:44.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc7162e5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc710107070 0x7fc71019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:44.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc7162e5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010c540 con 0x7fc710101e10 2024-06-18T13:02:44.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc7162e5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010c3c0 con 0x7fc710107070 2024-06-18T13:02:44.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc7162e5640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc71010c6c0 con 0x7fc710104740 2024-06-18T13:02:44.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc714882640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc710107070 0x7fc71019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:44.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc714882640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc710107070 0x7fc71019e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49524/0 (socket says 172.21.15.179:49524) 2024-06-18T13:02:44.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc714882640 1 -- 172.21.15.179:0/3511856681 learned_addr learned my addr 172.21.15.179:0/3511856681 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T13:02:44.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc70ffff640 1 --2- 172.21.15.179:0/3511856681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc710101e10 0x7fc710196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:44.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc707fff640 1 --2- 172.21.15.179:0/3511856681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc710104740 0x7fc7101970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:44.307 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc714882640 1 -- 172.21.15.179:0/3511856681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc710104740 msgr2=0x7fc7101970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:44.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc714882640 1 --2- 172.21.15.179:0/3511856681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc710104740 0x7fc7101970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:44.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc714882640 1 -- 172.21.15.179:0/3511856681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc710101e10 msgr2=0x7fc710196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:44.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc714882640 1 --2- 172.21.15.179:0/3511856681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc710101e10 0x7fc710196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:44.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.304+0000 7fc714882640 1 -- 172.21.15.179:0/3511856681 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71019e790 con 0x7fc710107070 2024-06-18T13:02:44.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.305+0000 7fc714882640 1 --2- 172.21.15.179:0/3511856681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc710107070 0x7fc71019e120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc70800cce0 tx=0x7fc70800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:44.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.305+0000 7fc70dffb640 1 -- 172.21.15.179:0/3511856681 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70800f450 con 0x7fc710107070 2024-06-18T13:02:44.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.305+0000 7fc70dffb640 1 -- 172.21.15.179:0/3511856681 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fc70800fa70 con 0x7fc710107070 2024-06-18T13:02:44.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.305+0000 7fc7162e5640 1 -- 172.21.15.179:0/3511856681 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71019ea80 con 0x7fc710107070 2024-06-18T13:02:44.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.305+0000 7fc70dffb640 1 -- 172.21.15.179:0/3511856681 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708018990 con 0x7fc710107070 2024-06-18T13:02:44.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.305+0000 7fc7162e5640 1 -- 172.21.15.179:0/3511856681 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71019ef90 con 0x7fc710107070 2024-06-18T13:02:44.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.306+0000 7fc7162e5640 1 -- 172.21.15.179:0/3511856681 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc710104b20 con 0x7fc710107070 2024-06-18T13:02:44.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.306+0000 7fc70dffb640 1 -- 172.21.15.179:0/3511856681 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) ==== 92117+0+0 (secure 0 0 0) 0x7fc70800f5f0 con 0x7fc710107070 2024-06-18T13:02:44.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.307+0000 7fc70dffb640 1 --2- 172.21.15.179:0/3511856681 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc6fc06df80 0x7fc6fc070440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:44.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.307+0000 7fc70dffb640 1 -- 172.21.15.179:0/3511856681 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7148+0+0 (secure 0 0 0) 0x7fc70808e6b0 con 0x7fc710107070 2024-06-18T13:02:44.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.307+0000 7fc70ffff640 1 --2- 172.21.15.179:0/3511856681 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc6fc06df80 0x7fc6fc070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:44.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.308+0000 7fc70ffff640 1 --2- 172.21.15.179:0/3511856681 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc6fc06df80 0x7fc6fc070440 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc70000abe0 tx=0x7fc70000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:44.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.311+0000 7fc70dffb640 1 -- 172.21.15.179:0/3511856681 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fc708014030 con 0x7fc710107070 2024-06-18T13:02:44.511 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.511+0000 7fc7162e5640 1 -- 172.21.15.179:0/3511856681 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi167=a;smithi179=b", "target": ["mon-mgr", ""]}) -- 0x7fc71002d650 con 0x7fc6fc06df80 2024-06-18T13:02:44.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.518+0000 7fc70dffb640 1 -- 172.21.15.179:0/3511856681 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+34 (secure 0 0 0) 0x7fc71002d650 con 0x7fc6fc06df80 2024-06-18T13:02:44.518 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled node-exporter update... 2024-06-18T13:02:44.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.520+0000 7fc7162e5640 1 -- 172.21.15.179:0/3511856681 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc6fc06df80 msgr2=0x7fc6fc070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:44.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.520+0000 7fc7162e5640 1 --2- 172.21.15.179:0/3511856681 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc6fc06df80 0x7fc6fc070440 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc70000abe0 tx=0x7fc70000a2d0 comp rx=0 tx=0).stop 2024-06-18T13:02:44.520 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.520+0000 7fc7162e5640 1 -- 172.21.15.179:0/3511856681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc710107070 msgr2=0x7fc71019e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:44.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.520+0000 7fc7162e5640 1 --2- 172.21.15.179:0/3511856681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc710107070 0x7fc71019e120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc70800cce0 tx=0x7fc70800a620 comp rx=0 tx=0).stop 2024-06-18T13:02:44.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.520+0000 7fc7162e5640 1 -- 172.21.15.179:0/3511856681 shutdown_connections 2024-06-18T13:02:44.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.520+0000 7fc7162e5640 1 --2- 172.21.15.179:0/3511856681 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fc6fc06df80 0x7fc6fc070440 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:44.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.520+0000 7fc7162e5640 1 --2- 172.21.15.179:0/3511856681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc710107070 0x7fc71019e120 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:44.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.520+0000 7fc7162e5640 1 --2- 172.21.15.179:0/3511856681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc710104740 0x7fc7101970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:44.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.520+0000 7fc7162e5640 1 --2- 172.21.15.179:0/3511856681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc710101e10 0x7fc710196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:44.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.520+0000 7fc7162e5640 1 -- 172.21.15.179:0/3511856681 >> 172.21.15.179:0/3511856681 conn(0x7fc7100fba30 msgr2=0x7fc710102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:44.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.520+0000 7fc7162e5640 1 -- 172.21.15.179:0/3511856681 shutdown_connections 2024-06-18T13:02:44.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:44.520+0000 7fc7162e5640 1 -- 172.21.15.179:0/3511856681 wait complete. 2024-06-18T13:02:44.769 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:44 smithi179 ceph-mon[33312]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-06-18T13:02:44.769 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:44 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:44.769 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:44 smithi179 ceph-mon[33312]: osdmap e70: 8 total, 8 up, 8 in 2024-06-18T13:02:44.769 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:44 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:44.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:44 smithi167 ceph-mon[29496]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-06-18T13:02:44.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:44 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:44 smithi167 ceph-mon[29496]: osdmap e70: 8 total, 8 up, 8 in 2024-06-18T13:02:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:44 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:44 smithi167 ceph-mon[37701]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-06-18T13:02:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:44 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:44 smithi167 ceph-mon[37701]: osdmap e70: 8 total, 8 up, 8 in 2024-06-18T13:02:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:44 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:45.267 DEBUG:teuthology.orchestra.run.smithi167:node-exporter.a> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@node-exporter.a.service 2024-06-18T13:02:45.270 DEBUG:teuthology.orchestra.run.smithi179:node-exporter.b> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@node-exporter.b.service 2024-06-18T13:02:45.272 INFO:tasks.cephadm:Adding alertmanager.a on smithi167 2024-06-18T13:02:45.273 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch apply alertmanager '1;smithi167=a' 2024-06-18T13:02:45.660 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:02:45.870 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:45 smithi179 ceph-mon[33312]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi167=a;smithi179=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:45.871 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:45 smithi179 ceph-mon[33312]: Saving service node-exporter spec with placement smithi167=a;smithi179=b;count:2 2024-06-18T13:02:45.871 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:45 smithi179 ceph-mon[33312]: mgrmap e16: y(active, since 4m), standbys: x 2024-06-18T13:02:45.871 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:45 smithi179 ceph-mon[33312]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 137 op/s 2024-06-18T13:02:45.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:45 smithi167 ceph-mon[29496]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi167=a;smithi179=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:45.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:45 smithi167 ceph-mon[29496]: Saving service node-exporter spec with placement smithi167=a;smithi179=b;count:2 2024-06-18T13:02:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:45 smithi167 ceph-mon[29496]: mgrmap e16: y(active, since 4m), standbys: x 2024-06-18T13:02:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:45 smithi167 ceph-mon[29496]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 137 op/s 2024-06-18T13:02:45.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:45 smithi167 ceph-mon[37701]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi167=a;smithi179=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:45.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:45 smithi167 ceph-mon[37701]: Saving service node-exporter spec with placement smithi167=a;smithi179=b;count:2 2024-06-18T13:02:45.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:45 smithi167 ceph-mon[37701]: mgrmap e16: y(active, since 4m), standbys: x 2024-06-18T13:02:45.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:45 smithi167 ceph-mon[37701]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 137 op/s 2024-06-18T13:02:47.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.215+0000 7f27dbc1e640 1 -- 172.21.15.179:0/3566520304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d41025b0 msgr2=0x7f27d4102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:47.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.215+0000 7f27dbc1e640 1 --2- 172.21.15.179:0/3566520304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d41025b0 0x7f27d4102a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009900 tx=0x7f27c402a410 comp rx=0 tx=0).stop 2024-06-18T13:02:47.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.215+0000 7f27dbc1e640 1 -- 172.21.15.179:0/3566520304 shutdown_connections 2024-06-18T13:02:47.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.215+0000 7f27dbc1e640 1 --2- 172.21.15.179:0/3566520304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27d4102f70 0x7f27d4109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:47.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.215+0000 7f27dbc1e640 1 --2- 172.21.15.179:0/3566520304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d41025b0 0x7f27d4102a30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:47.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.215+0000 7f27dbc1e640 1 --2- 172.21.15.179:0/3566520304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27d41013b0 0x7f27d41017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:47.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.215+0000 7f27dbc1e640 1 -- 172.21.15.179:0/3566520304 >> 172.21.15.179:0/3566520304 conn(0x7f27d40fcb40 msgr2=0x7f27d40fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:47.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.216+0000 7f27dbc1e640 1 -- 172.21.15.179:0/3566520304 shutdown_connections 2024-06-18T13:02:47.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.216+0000 7f27dbc1e640 1 -- 172.21.15.179:0/3566520304 wait complete. 2024-06-18T13:02:47.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.216+0000 7f27dbc1e640 1 Processor -- start 2024-06-18T13:02:47.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.216+0000 7f27dbc1e640 1 -- start start 2024-06-18T13:02:47.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.216+0000 7f27dbc1e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27d41013b0 0x7f27d4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:47.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.216+0000 7f27dbc1e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27d41025b0 0x7f27d4073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:47.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.216+0000 7f27dbc1e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d4102f70 0x7f27d40742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:47.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.216+0000 7f27dbc1e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27d410c520 con 0x7f27d41025b0 2024-06-18T13:02:47.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.216+0000 7f27dbc1e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f27d410c3a0 con 0x7f27d4102f70 2024-06-18T13:02:47.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.216+0000 7f27dbc1e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f27d410c6a0 con 0x7f27d41013b0 2024-06-18T13:02:47.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.216+0000 7f27da1bb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d4102f70 0x7f27d40742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:47.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.216+0000 7f27da1bb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d4102f70 0x7f27d40742d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49534/0 (socket says 172.21.15.179:49534) 2024-06-18T13:02:47.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.216+0000 7f27da1bb640 1 -- 172.21.15.179:0/2902934016 learned_addr learned my addr 172.21.15.179:0/2902934016 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T13:02:47.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.217+0000 7f27d91b9640 1 --2- 172.21.15.179:0/2902934016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27d41025b0 0x7f27d4073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:47.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.217+0000 7f27d99ba640 1 --2- 172.21.15.179:0/2902934016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27d41013b0 0x7f27d4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:47.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.217+0000 7f27da1bb640 1 -- 172.21.15.179:0/2902934016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27d41013b0 msgr2=0x7f27d4075740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:47.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.217+0000 7f27da1bb640 1 --2- 172.21.15.179:0/2902934016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27d41013b0 0x7f27d4075740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:47.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.217+0000 7f27da1bb640 1 -- 172.21.15.179:0/2902934016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27d41025b0 msgr2=0x7f27d4073d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:47.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.217+0000 7f27da1bb640 1 --2- 172.21.15.179:0/2902934016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27d41025b0 0x7f27d4073d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:47.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.217+0000 7f27da1bb640 1 -- 172.21.15.179:0/2902934016 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d4074b90 con 0x7f27d4102f70 2024-06-18T13:02:47.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.217+0000 7f27da1bb640 1 --2- 172.21.15.179:0/2902934016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d4102f70 0x7f27d40742d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00cad0 tx=0x7f27cc00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:47.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.217+0000 7f27caffd640 1 -- 172.21.15.179:0/2902934016 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc00ae20 con 0x7f27d4102f70 2024-06-18T13:02:47.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.217+0000 7f27dbc1e640 1 -- 172.21.15.179:0/2902934016 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d41a0dd0 con 0x7f27d4102f70 2024-06-18T13:02:47.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.217+0000 7f27caffd640 1 -- 172.21.15.179:0/2902934016 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f27cc005020 con 0x7f27d4102f70 2024-06-18T13:02:47.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.217+0000 7f27caffd640 1 -- 172.21.15.179:0/2902934016 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc00f6b0 con 0x7f27d4102f70 2024-06-18T13:02:47.219 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.217+0000 7f27dbc1e640 1 -- 172.21.15.179:0/2902934016 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d41a1230 con 0x7f27d4102f70 2024-06-18T13:02:47.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.218+0000 7f27dbc1e640 1 -- 172.21.15.179:0/2902934016 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d41017b0 con 0x7f27d4102f70 2024-06-18T13:02:47.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.219+0000 7f27caffd640 1 -- 172.21.15.179:0/2902934016 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 92168+0+0 (secure 0 0 0) 0x7f27cc004b20 con 0x7f27d4102f70 2024-06-18T13:02:47.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.219+0000 7f27caffd640 1 --2- 172.21.15.179:0/2902934016 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f27c006dfe0 0x7f27c00704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:47.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.219+0000 7f27caffd640 1 -- 172.21.15.179:0/2902934016 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7148+0+0 (secure 0 0 0) 0x7f27cc08e520 con 0x7f27d4102f70 2024-06-18T13:02:47.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.219+0000 7f27d99ba640 1 --2- 172.21.15.179:0/2902934016 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f27c006dfe0 0x7f27c00704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:47.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.220+0000 7f27d99ba640 1 --2- 172.21.15.179:0/2902934016 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f27c006dfe0 0x7f27c00704a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f27d4102410 tx=0x7f27d000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:47.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.224+0000 7f27caffd640 1 -- 172.21.15.179:0/2902934016 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f27cc014030 con 0x7f27d4102f70 2024-06-18T13:02:47.429 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.429+0000 7f27dbc1e640 1 -- 172.21.15.179:0/2902934016 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi167=a", "target": ["mon-mgr", ""]}) -- 0x7f27d402d5a0 con 0x7f27c006dfe0 2024-06-18T13:02:47.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.436+0000 7f27caffd640 1 -- 172.21.15.179:0/2902934016 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+33 (secure 0 0 0) 0x7f27d402d5a0 con 0x7f27c006dfe0 2024-06-18T13:02:47.436 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled alertmanager update... 2024-06-18T13:02:47.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.438+0000 7f27dbc1e640 1 -- 172.21.15.179:0/2902934016 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f27c006dfe0 msgr2=0x7f27c00704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:47.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.438+0000 7f27dbc1e640 1 --2- 172.21.15.179:0/2902934016 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f27c006dfe0 0x7f27c00704a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f27d4102410 tx=0x7f27d000a2d0 comp rx=0 tx=0).stop 2024-06-18T13:02:47.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.438+0000 7f27dbc1e640 1 -- 172.21.15.179:0/2902934016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d4102f70 msgr2=0x7f27d40742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:47.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.438+0000 7f27dbc1e640 1 --2- 172.21.15.179:0/2902934016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d4102f70 0x7f27d40742d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00cad0 tx=0x7f27cc00cf90 comp rx=0 tx=0).stop 2024-06-18T13:02:47.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.438+0000 7f27dbc1e640 1 -- 172.21.15.179:0/2902934016 shutdown_connections 2024-06-18T13:02:47.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.438+0000 7f27dbc1e640 1 --2- 172.21.15.179:0/2902934016 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f27c006dfe0 0x7f27c00704a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:47.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.438+0000 7f27dbc1e640 1 --2- 172.21.15.179:0/2902934016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d4102f70 0x7f27d40742d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:47.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.438+0000 7f27dbc1e640 1 --2- 172.21.15.179:0/2902934016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27d41025b0 0x7f27d4073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:47.438 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.438+0000 7f27dbc1e640 1 --2- 172.21.15.179:0/2902934016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27d41013b0 0x7f27d4075740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:47.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.438+0000 7f27dbc1e640 1 -- 172.21.15.179:0/2902934016 >> 172.21.15.179:0/2902934016 conn(0x7f27d40fcb40 msgr2=0x7f27d4105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:47.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.438+0000 7f27dbc1e640 1 -- 172.21.15.179:0/2902934016 shutdown_connections 2024-06-18T13:02:47.439 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:47.438+0000 7f27dbc1e640 1 -- 172.21.15.179:0/2902934016 wait complete. 2024-06-18T13:02:48.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:48 smithi179 ceph-mon[33312]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi167=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:48.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:48 smithi179 ceph-mon[33312]: Saving service alertmanager spec with placement smithi167=a;count:1 2024-06-18T13:02:48.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:48 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:48.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:48 smithi179 ceph-mon[33312]: pgmap v197: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-06-18T13:02:48.474 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:48 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:48.474 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:48 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:48.474 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:48 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:48.474 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:48 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:48.474 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:48 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:48.502 DEBUG:teuthology.orchestra.run.smithi167:alertmanager.a> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@alertmanager.a.service 2024-06-18T13:02:48.505 INFO:tasks.cephadm:Adding grafana.a on smithi179 2024-06-18T13:02:48.505 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph orch apply grafana '1;smithi179=a' 2024-06-18T13:02:48.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[37701]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi167=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[37701]: Saving service alertmanager spec with placement smithi167=a;count:1 2024-06-18T13:02:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[37701]: pgmap v197: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-06-18T13:02:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[29496]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi167=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[29496]: Saving service alertmanager spec with placement smithi167=a;count:1 2024-06-18T13:02:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[29496]: pgmap v197: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-06-18T13:02:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:02:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:02:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:48 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:48.766 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:02:49.872 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:49 smithi179 ceph-mon[33312]: Deploying daemon prometheus.a on smithi179 2024-06-18T13:02:49.872 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:49 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:49.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:49 smithi167 ceph-mon[37701]: Deploying daemon prometheus.a on smithi179 2024-06-18T13:02:49.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:49 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:49 smithi167 ceph-mon[29496]: Deploying daemon prometheus.a on smithi179 2024-06-18T13:02:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:49 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:50.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.643+0000 7fbdcff0a640 1 -- 172.21.15.179:0/1651162478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdc80ff6c0 msgr2=0x7fbdc80ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:50.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.643+0000 7fbdcff0a640 1 --2- 172.21.15.179:0/1651162478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdc80ff6c0 0x7fbdc80ffac0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009900 tx=0x7fbdc402a410 comp rx=0 tx=0).stop 2024-06-18T13:02:50.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.643+0000 7fbdcff0a640 1 -- 172.21.15.179:0/1651162478 shutdown_connections 2024-06-18T13:02:50.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.643+0000 7fbdcff0a640 1 --2- 172.21.15.179:0/1651162478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdc8103060 0x7fbdc81098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:50.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.643+0000 7fbdcff0a640 1 --2- 172.21.15.179:0/1651162478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc81026a0 0x7fbdc8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:50.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.643+0000 7fbdcff0a640 1 --2- 172.21.15.179:0/1651162478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdc80ff6c0 0x7fbdc80ffac0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:50.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.643+0000 7fbdcff0a640 1 -- 172.21.15.179:0/1651162478 >> 172.21.15.179:0/1651162478 conn(0x7fbdc80fb230 msgr2=0x7fbdc80fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:50.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.643+0000 7fbdcff0a640 1 -- 172.21.15.179:0/1651162478 shutdown_connections 2024-06-18T13:02:50.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.643+0000 7fbdcff0a640 1 -- 172.21.15.179:0/1651162478 wait complete. 2024-06-18T13:02:50.713 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcff0a640 1 Processor -- start 2024-06-18T13:02:50.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcff0a640 1 -- start start 2024-06-18T13:02:50.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcff0a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc80ff6c0 0x7fbdc8102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:50.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcff0a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdc81026a0 0x7fbdc8100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:50.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcff0a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdc8103060 0x7fbdc8100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:50.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcff0a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc810c4e0 con 0x7fbdc80ff6c0 2024-06-18T13:02:50.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcff0a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc810c360 con 0x7fbdc81026a0 2024-06-18T13:02:50.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcff0a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc810c660 con 0x7fbdc8103060 2024-06-18T13:02:50.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcd4a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdc81026a0 0x7fbdc8100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:50.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcd4a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdc81026a0 0x7fbdc8100650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59160/0 (socket says 172.21.15.179:59160) 2024-06-18T13:02:50.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcd4a5640 1 -- 172.21.15.179:0/1303662059 learned_addr learned my addr 172.21.15.179:0/1303662059 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T13:02:50.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdce4a7640 1 --2- 172.21.15.179:0/1303662059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdc8103060 0x7fbdc8100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:50.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcdca6640 1 --2- 172.21.15.179:0/1303662059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc80ff6c0 0x7fbdc8102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:50.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcd4a5640 1 -- 172.21.15.179:0/1303662059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdc8103060 msgr2=0x7fbdc8100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:50.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcd4a5640 1 --2- 172.21.15.179:0/1303662059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdc8103060 0x7fbdc8100b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:50.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcd4a5640 1 -- 172.21.15.179:0/1303662059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc80ff6c0 msgr2=0x7fbdc8102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:50.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcd4a5640 1 --2- 172.21.15.179:0/1303662059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc80ff6c0 0x7fbdc8102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:50.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.644+0000 7fbdcd4a5640 1 -- 172.21.15.179:0/1303662059 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc8101450 con 0x7fbdc81026a0 2024-06-18T13:02:50.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.645+0000 7fbdcd4a5640 1 --2- 172.21.15.179:0/1303662059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdc81026a0 0x7fbdc8100650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800d950 tx=0x7fbdb800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:50.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.645+0000 7fbdbeffd640 1 -- 172.21.15.179:0/1303662059 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8017070 con 0x7fbdc81026a0 2024-06-18T13:02:50.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.645+0000 7fbdbeffd640 1 -- 172.21.15.179:0/1303662059 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fbdb8009c30 con 0x7fbdc81026a0 2024-06-18T13:02:50.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.645+0000 7fbdcff0a640 1 -- 172.21.15.179:0/1303662059 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc81a0db0 con 0x7fbdc81026a0 2024-06-18T13:02:50.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.645+0000 7fbdbeffd640 1 -- 172.21.15.179:0/1303662059 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb800f620 con 0x7fbdc81026a0 2024-06-18T13:02:50.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.645+0000 7fbdcff0a640 1 -- 172.21.15.179:0/1303662059 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc81a1260 con 0x7fbdc81026a0 2024-06-18T13:02:50.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.646+0000 7fbdbcff9640 1 -- 172.21.15.179:0/1303662059 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb0005080 con 0x7fbdc81026a0 2024-06-18T13:02:50.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.647+0000 7fbdbeffd640 1 -- 172.21.15.179:0/1303662059 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 92168+0+0 (secure 0 0 0) 0x7fbdb8010460 con 0x7fbdc81026a0 2024-06-18T13:02:50.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.647+0000 7fbdbeffd640 1 --2- 172.21.15.179:0/1303662059 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fbdb406dfe0 0x7fbdb40704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:50.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.647+0000 7fbdbeffd640 1 -- 172.21.15.179:0/1303662059 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7148+0+0 (secure 0 0 0) 0x7fbdb808e080 con 0x7fbdc81026a0 2024-06-18T13:02:50.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.647+0000 7fbdcdca6640 1 --2- 172.21.15.179:0/1303662059 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fbdb406dfe0 0x7fbdb40704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:50.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.648+0000 7fbdcdca6640 1 --2- 172.21.15.179:0/1303662059 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fbdb406dfe0 0x7fbdb40704a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40057c0 tx=0x7fbdc40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:50.716 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.656+0000 7fbdbeffd640 1 -- 172.21.15.179:0/1303662059 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fbdb805aec0 con 0x7fbdc81026a0 2024-06-18T13:02:50.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:50 smithi179 ceph-mon[33312]: pgmap v198: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 2.8 KiB/s wr, 88 op/s 2024-06-18T13:02:50.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.920+0000 7fbdbcff9640 1 -- 172.21.15.179:0/1303662059 --> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}) -- 0x7fbdb0000bc0 con 0x7fbdb406dfe0 2024-06-18T13:02:50.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.928+0000 7fbdbeffd640 1 -- 172.21.15.179:0/1303662059 <== mgr.14150 v2:172.21.15.167:6800/1145517801 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fbdb0000bc0 con 0x7fbdb406dfe0 2024-06-18T13:02:50.928 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled grafana update... 2024-06-18T13:02:50.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.931+0000 7fbdbcff9640 1 -- 172.21.15.179:0/1303662059 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fbdb406dfe0 msgr2=0x7fbdb40704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:50.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.931+0000 7fbdbcff9640 1 --2- 172.21.15.179:0/1303662059 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fbdb406dfe0 0x7fbdb40704a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40057c0 tx=0x7fbdc40058d0 comp rx=0 tx=0).stop 2024-06-18T13:02:50.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.931+0000 7fbdbcff9640 1 -- 172.21.15.179:0/1303662059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdc81026a0 msgr2=0x7fbdc8100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:50.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.931+0000 7fbdbcff9640 1 --2- 172.21.15.179:0/1303662059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdc81026a0 0x7fbdc8100650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800d950 tx=0x7fbdb800de10 comp rx=0 tx=0).stop 2024-06-18T13:02:50.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.931+0000 7fbdbcff9640 1 -- 172.21.15.179:0/1303662059 shutdown_connections 2024-06-18T13:02:50.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.931+0000 7fbdbcff9640 1 --2- 172.21.15.179:0/1303662059 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fbdb406dfe0 0x7fbdb40704a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:50.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.931+0000 7fbdbcff9640 1 --2- 172.21.15.179:0/1303662059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdc8103060 0x7fbdc8100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:50.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.931+0000 7fbdbcff9640 1 --2- 172.21.15.179:0/1303662059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdc81026a0 0x7fbdc8100650 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:50.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.931+0000 7fbdbcff9640 1 --2- 172.21.15.179:0/1303662059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc80ff6c0 0x7fbdc8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:50.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.931+0000 7fbdbcff9640 1 -- 172.21.15.179:0/1303662059 >> 172.21.15.179:0/1303662059 conn(0x7fbdc80fb230 msgr2=0x7fbdc80fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:50.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.931+0000 7fbdbcff9640 1 -- 172.21.15.179:0/1303662059 shutdown_connections 2024-06-18T13:02:50.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:50.931+0000 7fbdbcff9640 1 -- 172.21.15.179:0/1303662059 wait complete. 2024-06-18T13:02:50.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:50 smithi167 ceph-mon[37701]: pgmap v198: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 2.8 KiB/s wr, 88 op/s 2024-06-18T13:02:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:50 smithi167 ceph-mon[29496]: pgmap v198: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 2.8 KiB/s wr, 88 op/s 2024-06-18T13:02:51.875 DEBUG:teuthology.orchestra.run.smithi179:grafana.a> sudo journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@grafana.a.service 2024-06-18T13:02:51.877 INFO:tasks.cephadm:Setting up client nodes... 2024-06-18T13:02:51.878 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-18T13:02:52.128 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:02:52.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:51 smithi179 ceph-mon[33312]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:52.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:51 smithi179 ceph-mon[33312]: Saving service grafana spec with placement smithi179=a;count:1 2024-06-18T13:02:52.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:51 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:52.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:51 smithi179 ceph-mon[33312]: pgmap v199: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1.1 KiB/s wr, 33 op/s 2024-06-18T13:02:52.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:51 smithi167 ceph-mon[37701]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:52.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:51 smithi167 ceph-mon[37701]: Saving service grafana spec with placement smithi179=a;count:1 2024-06-18T13:02:52.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:51 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:52.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:51 smithi167 ceph-mon[37701]: pgmap v199: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1.1 KiB/s wr, 33 op/s 2024-06-18T13:02:52.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:51 smithi167 ceph-mon[29496]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-18T13:02:52.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:51 smithi167 ceph-mon[29496]: Saving service grafana spec with placement smithi179=a;count:1 2024-06-18T13:02:52.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:51 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:02:52.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:51 smithi167 ceph-mon[29496]: pgmap v199: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1.1 KiB/s wr, 33 op/s 2024-06-18T13:02:52.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:02:52 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:02:53.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:52 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:02:53.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:52 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:02:53.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:52 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:02:53.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.785+0000 7fd7ba50e640 1 -- 172.21.15.167:0/3911885954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b4101e80 msgr2=0x7fd7b4108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:53.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.785+0000 7fd7ba50e640 1 --2- 172.21.15.167:0/3911885954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b4101e80 0x7fd7b4108710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009ca0 tx=0x7fd7ac02a8c0 comp rx=0 tx=0).stop 2024-06-18T13:02:53.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.785+0000 7fd7ba50e640 1 -- 172.21.15.167:0/3911885954 shutdown_connections 2024-06-18T13:02:53.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.785+0000 7fd7ba50e640 1 --2- 172.21.15.167:0/3911885954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b4101e80 0x7fd7b4108710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:53.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.785+0000 7fd7ba50e640 1 --2- 172.21.15.167:0/3911885954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7b41014c0 0x7fd7b4101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:53.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.785+0000 7fd7ba50e640 1 --2- 172.21.15.167:0/3911885954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b41002c0 0x7fd7b41006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:53.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.785+0000 7fd7ba50e640 1 -- 172.21.15.167:0/3911885954 >> 172.21.15.167:0/3911885954 conn(0x7fd7b40fba50 msgr2=0x7fd7b40fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:53.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.785+0000 7fd7ba50e640 1 -- 172.21.15.167:0/3911885954 shutdown_connections 2024-06-18T13:02:53.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.785+0000 7fd7ba50e640 1 -- 172.21.15.167:0/3911885954 wait complete. 2024-06-18T13:02:53.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.786+0000 7fd7ba50e640 1 Processor -- start 2024-06-18T13:02:53.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.786+0000 7fd7ba50e640 1 -- start start 2024-06-18T13:02:53.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.786+0000 7fd7ba50e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7b41002c0 0x7fd7b406aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.786+0000 7fd7ba50e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b41014c0 0x7fd7b419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.786+0000 7fd7ba50e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b4101e80 0x7fd7b419df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.786+0000 7fd7ba50e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b410b430 con 0x7fd7b4101e80 2024-06-18T13:02:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.786+0000 7fd7ba50e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b410b2b0 con 0x7fd7b41002c0 2024-06-18T13:02:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.786+0000 7fd7ba50e640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b410b5b0 con 0x7fd7b41014c0 2024-06-18T13:02:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.787+0000 7fd7b37fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b41014c0 0x7fd7b419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.787+0000 7fd7b8aab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b4101e80 0x7fd7b419df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.787+0000 7fd7b3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7b41002c0 0x7fd7b406aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.787+0000 7fd7b37fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b41014c0 0x7fd7b419b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:51144/0 (socket says 172.21.15.167:51144) 2024-06-18T13:02:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.787+0000 7fd7b3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7b41002c0 0x7fd7b406aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.167:59156/0 (socket says 172.21.15.167:59156) 2024-06-18T13:02:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.787+0000 7fd7b37fe640 1 -- 172.21.15.167:0/2852534960 learned_addr learned my addr 172.21.15.167:0/2852534960 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:02:53.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.787+0000 7fd7b3fff640 1 -- 172.21.15.167:0/2852534960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b41014c0 msgr2=0x7fd7b419b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.787+0000 7fd7b3fff640 1 --2- 172.21.15.167:0/2852534960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b41014c0 0x7fd7b419b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.787+0000 7fd7b3fff640 1 -- 172.21.15.167:0/2852534960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b4101e80 msgr2=0x7fd7b419df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.787+0000 7fd7b3fff640 1 --2- 172.21.15.167:0/2852534960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b4101e80 0x7fd7b419df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.787+0000 7fd7b3fff640 1 -- 172.21.15.167:0/2852534960 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b419e810 con 0x7fd7b41002c0 2024-06-18T13:02:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.787+0000 7fd7b8aab640 1 --2- 172.21.15.167:0/2852534960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b4101e80 0x7fd7b419df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:02:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.787+0000 7fd7b3fff640 1 --2- 172.21.15.167:0/2852534960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7b41002c0 0x7fd7b406aaa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400e9c0 tx=0x7fd7a400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.788+0000 7fd7b17fa640 1 -- 172.21.15.167:0/2852534960 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a400cd80 con 0x7fd7b41002c0 2024-06-18T13:02:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.788+0000 7fd7ba50e640 1 -- 172.21.15.167:0/2852534960 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b419eaa0 con 0x7fd7b41002c0 2024-06-18T13:02:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.788+0000 7fd7b17fa640 1 -- 172.21.15.167:0/2852534960 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fd7a40108b0 con 0x7fd7b41002c0 2024-06-18T13:02:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.788+0000 7fd7b17fa640 1 -- 172.21.15.167:0/2852534960 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4018720 con 0x7fd7b41002c0 2024-06-18T13:02:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.788+0000 7fd7ba50e640 1 -- 172.21.15.167:0/2852534960 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b419f030 con 0x7fd7b41002c0 2024-06-18T13:02:53.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.789+0000 7fd7ba50e640 1 -- 172.21.15.167:0/2852534960 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b41006c0 con 0x7fd7b41002c0 2024-06-18T13:02:53.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.789+0000 7fd7b17fa640 1 -- 172.21.15.167:0/2852534960 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 92168+0+0 (secure 0 0 0) 0x7fd7a401f020 con 0x7fd7b41002c0 2024-06-18T13:02:53.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.790+0000 7fd7b17fa640 1 --2- 172.21.15.167:0/2852534960 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fd79c06dfe0 0x7fd79c0704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:53.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.790+0000 7fd7b37fe640 1 --2- 172.21.15.167:0/2852534960 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fd79c06dfe0 0x7fd79c0704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:53.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.790+0000 7fd7b17fa640 1 -- 172.21.15.167:0/2852534960 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7148+0+0 (secure 0 0 0) 0x7fd7a408e5c0 con 0x7fd7b41002c0 2024-06-18T13:02:53.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.790+0000 7fd7b37fe640 1 --2- 172.21.15.167:0/2852534960 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fd79c06dfe0 0x7fd79c0704a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd7b406b020 tx=0x7fd7a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:53.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:53.798+0000 7fd7b17fa640 1 -- 172.21.15.167:0/2852534960 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fd7a405b280 con 0x7fd7b41002c0 2024-06-18T13:02:54.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:54.045+0000 7fd7ba50e640 1 -- 172.21.15.167:0/2852534960 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd7b402d2e0 con 0x7fd7b41002c0 2024-06-18T13:02:54.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:54.052+0000 7fd7b17fa640 1 -- 172.21.15.167:0/2852534960 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) ==== 170+0+59 (secure 0 0 0) 0x7fd7a405f580 con 0x7fd7b41002c0 2024-06-18T13:02:54.053 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2024-06-18T13:02:54.053 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQB+hXFmrG/WAhAAUqjSXXUQZ3z3eQnYS6wIuQ== 2024-06-18T13:02:54.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:54.054+0000 7fd7ba50e640 1 -- 172.21.15.167:0/2852534960 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fd79c06dfe0 msgr2=0x7fd79c0704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:54.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:54.054+0000 7fd7ba50e640 1 --2- 172.21.15.167:0/2852534960 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fd79c06dfe0 0x7fd79c0704a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd7b406b020 tx=0x7fd7a8005f90 comp rx=0 tx=0).stop 2024-06-18T13:02:54.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:54.054+0000 7fd7ba50e640 1 -- 172.21.15.167:0/2852534960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7b41002c0 msgr2=0x7fd7b406aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:54.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:54.054+0000 7fd7ba50e640 1 --2- 172.21.15.167:0/2852534960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7b41002c0 0x7fd7b406aaa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400e9c0 tx=0x7fd7a400ee80 comp rx=0 tx=0).stop 2024-06-18T13:02:54.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:54.054+0000 7fd7ba50e640 1 -- 172.21.15.167:0/2852534960 shutdown_connections 2024-06-18T13:02:54.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:54.054+0000 7fd7ba50e640 1 --2- 172.21.15.167:0/2852534960 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fd79c06dfe0 0x7fd79c0704a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:54.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:54.054+0000 7fd7ba50e640 1 --2- 172.21.15.167:0/2852534960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b4101e80 0x7fd7b419df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:54.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:54.055+0000 7fd7ba50e640 1 --2- 172.21.15.167:0/2852534960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b41014c0 0x7fd7b419b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:54.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:54.055+0000 7fd7ba50e640 1 --2- 172.21.15.167:0/2852534960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7b41002c0 0x7fd7b406aaa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:54.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:54.055+0000 7fd7ba50e640 1 -- 172.21.15.167:0/2852534960 >> 172.21.15.167:0/2852534960 conn(0x7fd7b40fba50 msgr2=0x7fd7b40fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:54.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:54.055+0000 7fd7ba50e640 1 -- 172.21.15.167:0/2852534960 shutdown_connections 2024-06-18T13:02:54.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:02:54.055+0000 7fd7ba50e640 1 -- 172.21.15.167:0/2852534960 wait complete. 2024-06-18T13:02:54.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:53 smithi167 ceph-mon[29496]: pgmap v200: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-06-18T13:02:54.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:53 smithi167 ceph-mon[37701]: pgmap v200: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-06-18T13:02:54.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:53 smithi179 ceph-mon[33312]: pgmap v200: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-06-18T13:02:54.718 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T13:02:54.718 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-18T13:02:54.718 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-18T13:02:54.758 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-18T13:02:54.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2852534960' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-18T13:02:54.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-18T13:02:54.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-18T13:02:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2852534960' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-18T13:02:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-18T13:02:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-18T13:02:55.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2852534960' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-18T13:02:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-18T13:02:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-18T13:02:55.634 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.b/config 2024-06-18T13:02:56.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:55 smithi179 ceph-mon[33312]: pgmap v201: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 857 B/s rd, 85 B/s wr, 1 op/s 2024-06-18T13:02:56.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:55 smithi167 ceph-mon[37701]: pgmap v201: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 857 B/s rd, 85 B/s wr, 1 op/s 2024-06-18T13:02:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:55 smithi167 ceph-mon[29496]: pgmap v201: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 857 B/s rd, 85 B/s wr, 1 op/s 2024-06-18T13:02:57.734 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:57 smithi179 ceph-mon[33312]: pgmap v202: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-06-18T13:02:57.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:57 smithi167 ceph-mon[37701]: pgmap v202: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-06-18T13:02:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:57 smithi167 ceph-mon[29496]: pgmap v202: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-06-18T13:02:59.811 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:02:59 smithi179 ceph-mon[33312]: pgmap v203: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:02:59.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.906+0000 7fecc87d8640 1 -- 172.21.15.179:0/1725979403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecc01025b0 msgr2=0x7fecc0102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:59.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.906+0000 7fecc87d8640 1 --2- 172.21.15.179:0/1725979403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecc01025b0 0x7fecc0102a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fecbc009900 tx=0x7fecbc02a410 comp rx=0 tx=0).stop 2024-06-18T13:02:59.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.906+0000 7fecc87d8640 1 -- 172.21.15.179:0/1725979403 shutdown_connections 2024-06-18T13:02:59.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.906+0000 7fecc87d8640 1 --2- 172.21.15.179:0/1725979403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecc0102f70 0x7fecc0109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:59.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.906+0000 7fecc87d8640 1 --2- 172.21.15.179:0/1725979403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecc01025b0 0x7fecc0102a30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:59.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.906+0000 7fecc87d8640 1 --2- 172.21.15.179:0/1725979403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecc01013b0 0x7fecc01017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:59.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.906+0000 7fecc87d8640 1 -- 172.21.15.179:0/1725979403 >> 172.21.15.179:0/1725979403 conn(0x7fecc00fcb40 msgr2=0x7fecc00fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:02:59.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.906+0000 7fecc87d8640 1 -- 172.21.15.179:0/1725979403 shutdown_connections 2024-06-18T13:02:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.906+0000 7fecc87d8640 1 -- 172.21.15.179:0/1725979403 wait complete. 2024-06-18T13:02:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.907+0000 7fecc87d8640 1 Processor -- start 2024-06-18T13:02:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.907+0000 7fecc87d8640 1 -- start start 2024-06-18T13:02:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.907+0000 7fecc87d8640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecc01013b0 0x7fecc0196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.907+0000 7fecc87d8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecc01025b0 0x7fecc0197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.907+0000 7fecc87d8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecc0102f70 0x7fecc019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.907+0000 7fecc87d8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fecc010c520 con 0x7fecc01025b0 2024-06-18T13:02:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.907+0000 7fecc87d8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fecc010c3a0 con 0x7fecc0102f70 2024-06-18T13:02:59.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.907+0000 7fecc87d8640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fecc010c6a0 con 0x7fecc01013b0 2024-06-18T13:02:59.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc6d75640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecc0102f70 0x7fecc019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc6d75640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecc0102f70 0x7fecc019e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59182/0 (socket says 172.21.15.179:59182) 2024-06-18T13:02:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc6d75640 1 -- 172.21.15.179:0/2898572242 learned_addr learned my addr 172.21.15.179:0/2898572242 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-18T13:02:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc6574640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecc01013b0 0x7fecc0196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc5d73640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecc01025b0 0x7fecc0197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc6d75640 1 -- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecc01013b0 msgr2=0x7fecc0196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc6d75640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecc01013b0 0x7fecc0196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc6d75640 1 -- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecc01025b0 msgr2=0x7fecc0197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:02:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc6d75640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecc01025b0 0x7fecc0197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:02:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc6d75640 1 -- 172.21.15.179:0/2898572242 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc019e800 con 0x7fecc0102f70 2024-06-18T13:02:59.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc5d73640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecc01025b0 0x7fecc0197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T13:02:59.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc6574640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecc01013b0 0x7fecc0196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T13:02:59.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc6d75640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecc0102f70 0x7fecc019e190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fecb800cce0 tx=0x7fecb800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:59.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecb77fe640 1 -- 172.21.15.179:0/2898572242 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb800f450 con 0x7fecc0102f70 2024-06-18T13:02:59.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecb77fe640 1 -- 172.21.15.179:0/2898572242 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fecb800fa70 con 0x7fecc0102f70 2024-06-18T13:02:59.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc87d8640 1 -- 172.21.15.179:0/2898572242 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc019eaf0 con 0x7fecc0102f70 2024-06-18T13:02:59.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecb77fe640 1 -- 172.21.15.179:0/2898572242 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8018990 con 0x7fecc0102f70 2024-06-18T13:02:59.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.908+0000 7fecc87d8640 1 -- 172.21.15.179:0/2898572242 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc0073bd0 con 0x7fecc0102f70 2024-06-18T13:02:59.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.909+0000 7fecc87d8640 1 -- 172.21.15.179:0/2898572242 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc01017b0 con 0x7fecc0102f70 2024-06-18T13:02:59.911 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.910+0000 7fecb77fe640 1 -- 172.21.15.179:0/2898572242 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 92168+0+0 (secure 0 0 0) 0x7fecb800f5f0 con 0x7fecc0102f70 2024-06-18T13:02:59.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.911+0000 7fecb77fe640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fecac072420 0x7fecac0748e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:02:59.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.911+0000 7fecb77fe640 1 -- 172.21.15.179:0/2898572242 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7148+0+0 (secure 0 0 0) 0x7fecb808e980 con 0x7fecc0102f70 2024-06-18T13:02:59.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.911+0000 7fecc6574640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fecac072420 0x7fecac0748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:02:59.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.911+0000 7fecc6574640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fecac072420 0x7fecac0748e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fecb00058c0 tx=0x7fecb0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:02:59.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:02:59.915+0000 7fecb77fe640 1 -- 172.21.15.179:0/2898572242 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fecb8014030 con 0x7fecc0102f70 2024-06-18T13:02:59.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:02:59 smithi167 ceph-mon[37701]: pgmap v203: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:02:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:02:59 smithi167 ceph-mon[29496]: pgmap v203: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:00.188 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:03:00.187+0000 7fecc87d8640 1 -- 172.21.15.179:0/2898572242 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fecc002d550 con 0x7fecc0102f70 2024-06-18T13:03:00.192 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:03:00.192+0000 7fecb77fe640 1 -- 172.21.15.179:0/2898572242 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) ==== 170+0+59 (secure 0 0 0) 0x7fecb805bd90 con 0x7fecc0102f70 2024-06-18T13:03:00.192 INFO:teuthology.orchestra.run.smithi179.stdout:[client.1] 2024-06-18T13:03:00.192 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCEhXFmaUVQCxAAhX7dPExnoU6BAWexf5Fr6w== 2024-06-18T13:03:00.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:03:00.194+0000 7fecc87d8640 1 -- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fecac072420 msgr2=0x7fecac0748e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:00.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:03:00.194+0000 7fecc87d8640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fecac072420 0x7fecac0748e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fecb00058c0 tx=0x7fecb0005850 comp rx=0 tx=0).stop 2024-06-18T13:03:00.194 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:03:00.194+0000 7fecc87d8640 1 -- 172.21.15.179:0/2898572242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecc0102f70 msgr2=0x7fecc019e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:00.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:03:00.195+0000 7fecc87d8640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecc0102f70 0x7fecc019e190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fecb800cce0 tx=0x7fecb800a620 comp rx=0 tx=0).stop 2024-06-18T13:03:00.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:03:00.195+0000 7fecc87d8640 1 -- 172.21.15.179:0/2898572242 shutdown_connections 2024-06-18T13:03:00.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:03:00.195+0000 7fecc87d8640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fecac072420 0x7fecac0748e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:00.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:03:00.195+0000 7fecc87d8640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecc0102f70 0x7fecc019e190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:00.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:03:00.195+0000 7fecc87d8640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecc01025b0 0x7fecc0197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:00.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:03:00.195+0000 7fecc87d8640 1 --2- 172.21.15.179:0/2898572242 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecc01013b0 0x7fecc0196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:00.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:03:00.195+0000 7fecc87d8640 1 -- 172.21.15.179:0/2898572242 >> 172.21.15.179:0/2898572242 conn(0x7fecc00fcb40 msgr2=0x7fecc01059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:00.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:03:00.195+0000 7fecc87d8640 1 -- 172.21.15.179:0/2898572242 shutdown_connections 2024-06-18T13:03:00.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-18T13:03:00.195+0000 7fecc87d8640 1 -- 172.21.15.179:0/2898572242 wait complete. 2024-06-18T13:03:00.792 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.179:0/2898572242' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-18T13:03:00.792 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-18T13:03:00.792 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-18T13:03:00.819 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-18T13:03:00.819 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-06-18T13:03:00.819 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-06-18T13:03:00.857 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-18T13:03:00.857 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-18T13:03:00.857 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph mgr dump --format=json 2024-06-18T13:03:00.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.179:0/2898572242' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-18T13:03:00.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-18T13:03:00.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-18T13:03:00.885 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.179:0/2898572242' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-18T13:03:00.885 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-18T13:03:00.885 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-18T13:03:01.105 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:01.636 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:03:01 smithi179 systemd[1]: Started Ceph prometheus.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:03:02.563 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ignoring --setuser ceph since I am not root 2024-06-18T13:03:02.563 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ignoring --setgroup ceph since I am not root 2024-06-18T13:03:02.564 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:02.460+0000 7f739c927040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-18T13:03:02.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:03:02.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:03:02.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:03:02.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-18T13:03:02.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-mon[37701]: pgmap v204: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:02.564 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:03:02.564 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:03:02.564 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:03:02.565 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-18T13:03:02.565 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-mon[29496]: pgmap v204: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:02.613 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:03:02.613 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:03:02.613 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' 2024-06-18T13:03:02.613 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:02 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-18T13:03:02.613 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:02 smithi179 ceph-mon[33312]: pgmap v204: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:02.613 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:02 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: ignoring --setuser ceph since I am not root 2024-06-18T13:03:02.614 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:02 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: ignoring --setgroup ceph since I am not root 2024-06-18T13:03:02.614 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:02 smithi179 ceph-mgr[35150]: -- 172.21.15.179:0/1590627844 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55de544f90e0 con 0x55de544d7000 2024-06-18T13:03:02.614 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:02 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:02.461+0000 7f7f60f9e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-18T13:03:02.614 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:02 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:02.613+0000 7f7f60f9e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-18T13:03:02.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.783+0000 7f50f0509640 1 -- 172.21.15.167:0/3426883189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e81025a0 msgr2=0x7f50e8102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:02.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.784+0000 7f50f0509640 1 --2- 172.21.15.167:0/3426883189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e81025a0 0x7f50e8102980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009900 tx=0x7f50e402a410 comp rx=0 tx=0).stop 2024-06-18T13:03:02.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.784+0000 7f50f0509640 1 -- 172.21.15.167:0/3426883189 shutdown_connections 2024-06-18T13:03:02.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.784+0000 7f50f0509640 1 --2- 172.21.15.167:0/3426883189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e810baa0 0x7f50e810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:02.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.784+0000 7f50f0509640 1 --2- 172.21.15.167:0/3426883189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e8102f50 0x7f50e810b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:02.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.784+0000 7f50f0509640 1 --2- 172.21.15.167:0/3426883189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e81025a0 0x7f50e8102980 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:02.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.784+0000 7f50f0509640 1 -- 172.21.15.167:0/3426883189 >> 172.21.15.167:0/3426883189 conn(0x7f50e80fbf80 msgr2=0x7f50e80fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:02.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.784+0000 7f50f0509640 1 -- 172.21.15.167:0/3426883189 shutdown_connections 2024-06-18T13:03:02.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.784+0000 7f50f0509640 1 -- 172.21.15.167:0/3426883189 wait complete. 2024-06-18T13:03:02.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.784+0000 7f50f0509640 1 Processor -- start 2024-06-18T13:03:02.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.784+0000 7f50f0509640 1 -- start start 2024-06-18T13:03:02.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50f0509640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e81025a0 0x7f50e8197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:02.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50f0509640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e8102f50 0x7f50e8197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:02.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50f0509640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e810baa0 0x7f50e819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:02.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50f0509640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f50e8110d30 con 0x7f50e81025a0 2024-06-18T13:03:02.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50f0509640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f50e8110bb0 con 0x7f50e8102f50 2024-06-18T13:03:02.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50f0509640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f50e8110eb0 con 0x7f50e810baa0 2024-06-18T13:03:02.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50edaa4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e8102f50 0x7f50e8197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:02.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50ee2a5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e81025a0 0x7f50e8197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:02.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50edaa4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e8102f50 0x7f50e8197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.167:59178/0 (socket says 172.21.15.167:59178) 2024-06-18T13:03:02.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50edaa4640 1 -- 172.21.15.167:0/1369831079 learned_addr learned my addr 172.21.15.167:0/1369831079 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:02.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50ee2a5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e81025a0 0x7f50e8197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35490/0 (socket says 172.21.15.167:35490) 2024-06-18T13:03:02.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50eeaa6640 1 --2- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e810baa0 0x7f50e819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:02.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50ee2a5640 1 -- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e810baa0 msgr2=0x7f50e819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:02.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50ee2a5640 1 --2- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e810baa0 0x7f50e819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:02.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50ee2a5640 1 -- 172.21.15.167:0/1369831079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e8102f50 msgr2=0x7f50e8197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:02.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50ee2a5640 1 --2- 172.21.15.167:0/1369831079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e8102f50 0x7f50e8197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:02.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50ee2a5640 1 -- 172.21.15.167:0/1369831079 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e819c2d0 con 0x7f50e81025a0 2024-06-18T13:03:02.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.785+0000 7f50ee2a5640 1 --2- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e81025a0 0x7f50e8197320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f50e402a930 tx=0x7f50e4004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:02.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.786+0000 7f50df7fe640 1 -- 172.21.15.167:0/1369831079 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e403f070 con 0x7f50e81025a0 2024-06-18T13:03:02.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.786+0000 7f50df7fe640 1 -- 172.21.15.167:0/1369831079 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f50e40054d0 con 0x7f50e81025a0 2024-06-18T13:03:02.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.786+0000 7f50f0509640 1 -- 172.21.15.167:0/1369831079 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e819c560 con 0x7f50e81025a0 2024-06-18T13:03:02.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.786+0000 7f50df7fe640 1 -- 172.21.15.167:0/1369831079 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4035030 con 0x7f50e81025a0 2024-06-18T13:03:02.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.786+0000 7f50f0509640 1 -- 172.21.15.167:0/1369831079 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e81a3ea0 con 0x7f50e81025a0 2024-06-18T13:03:02.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.787+0000 7f50f0509640 1 -- 172.21.15.167:0/1369831079 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50e8102980 con 0x7f50e81025a0 2024-06-18T13:03:02.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.787+0000 7f50df7fe640 1 -- 172.21.15.167:0/1369831079 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 92182+0+0 (secure 0 0 0) 0x7f50e4042070 con 0x7f50e81025a0 2024-06-18T13:03:02.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.788+0000 7f50df7fe640 1 --2- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f50d406e030 0x7f50d40704f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:02.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.788+0000 7f50edaa4640 1 -- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f50d406e030 msgr2=0x7f50d40704f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/1145517801 2024-06-18T13:03:02.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.788+0000 7f50df7fe640 1 -- 172.21.15.167:0/1369831079 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7148+0+0 (secure 0 0 0) 0x7f50e40aeb00 con 0x7f50e81025a0 2024-06-18T13:03:02.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.788+0000 7f50edaa4640 1 --2- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f50d406e030 0x7f50d40704f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-18T13:03:02.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.792+0000 7f50df7fe640 1 -- 172.21.15.167:0/1369831079 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f50e407b940 con 0x7f50e81025a0 2024-06-18T13:03:02.886 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:02 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:02.681+0000 7f7f60f9e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-18T13:03:02.942 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:02.616+0000 7f739c927040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-18T13:03:02.943 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:02.687+0000 7f739c927040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-18T13:03:02.943 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:02.942+0000 7f739c927040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-18T13:03:02.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.989+0000 7f50edaa4640 1 -- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f50d406e030 msgr2=0x7f50d40704f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/1145517801 2024-06-18T13:03:02.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:02.989+0000 7f50edaa4640 1 --2- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f50d406e030 0x7f50d40704f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-18T13:03:03.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:03.017+0000 7f50f0509640 1 -- 172.21.15.167:0/1369831079 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f50e81a4070 con 0x7f50e81025a0 2024-06-18T13:03:03.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:03.020+0000 7f50df7fe640 1 -- 172.21.15.167:0/1369831079 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+176023 (secure 0 0 0) 0x7f50e407fc40 con 0x7f50e81025a0 2024-06-18T13:03:03.023 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T13:03:03.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:03.023+0000 7f50f0509640 1 -- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f50d406e030 msgr2=0x7f50d40704f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-18T13:03:03.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:03.023+0000 7f50f0509640 1 --2- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f50d406e030 0x7f50d40704f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:03.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:03.023+0000 7f50f0509640 1 -- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e81025a0 msgr2=0x7f50e8197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:03.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:03.023+0000 7f50f0509640 1 --2- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e81025a0 0x7f50e8197320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f50e402a930 tx=0x7f50e4004e80 comp rx=0 tx=0).stop 2024-06-18T13:03:03.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:03.023+0000 7f50f0509640 1 -- 172.21.15.167:0/1369831079 shutdown_connections 2024-06-18T13:03:03.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:03.023+0000 7f50f0509640 1 --2- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7f50d406e030 0x7f50d40704f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:03.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:03.023+0000 7f50f0509640 1 --2- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e810baa0 0x7f50e819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:03.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:03.023+0000 7f50f0509640 1 --2- 172.21.15.167:0/1369831079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50e8102f50 0x7f50e8197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:03.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:03.023+0000 7f50f0509640 1 --2- 172.21.15.167:0/1369831079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e81025a0 0x7f50e8197320 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:03.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:03.023+0000 7f50f0509640 1 -- 172.21.15.167:0/1369831079 >> 172.21.15.167:0/1369831079 conn(0x7f50e80fbf80 msgr2=0x7f50e810d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:03.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:03.023+0000 7f50f0509640 1 -- 172.21.15.167:0/1369831079 shutdown_connections 2024-06-18T13:03:03.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:03.023+0000 7f50f0509640 1 -- 172.21.15.167:0/1369831079 wait complete. 2024-06-18T13:03:03.226 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:03 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:03.004+0000 7f739c927040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-18T13:03:03.302 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:02 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:02.926+0000 7f7f60f9e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-18T13:03:03.302 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:02 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:02.988+0000 7f7f60f9e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-18T13:03:03.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:03 smithi179 ceph-mon[33312]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-18T13:03:03.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:03 smithi179 ceph-mon[33312]: mgrmap e17: y(active, since 4m), standbys: x 2024-06-18T13:03:03.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:03 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1369831079' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-18T13:03:03.706 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:03 smithi167 ceph-mon[37701]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-18T13:03:03.706 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:03 smithi167 ceph-mon[37701]: mgrmap e17: y(active, since 4m), standbys: x 2024-06-18T13:03:03.706 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:03 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1369831079' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-18T13:03:03.706 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:03 smithi167 ceph-mon[29496]: from='mgr.14150 172.21.15.167:0/4043023007' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-18T13:03:03.706 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:03 smithi167 ceph-mon[29496]: mgrmap e17: y(active, since 4m), standbys: x 2024-06-18T13:03:03.706 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:03 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1369831079' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-18T13:03:03.734 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1145517801},{"type":"v1","addr":"172.21.15.167:6801","nonce":1145517801}]},"active_addr":"172.21.15.167:6801/1145517801","active_change":"2024-06-18T12:58:13.490505+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.167:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":1266240186}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":506888618}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":3362690351}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":134973485}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":499306102}]}]} 2024-06-18T13:03:03.740 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-18T13:03:03.740 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-18T13:03:03.740 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd dump --format=json 2024-06-18T13:03:04.019 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:04.262 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:03 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:03.977+0000 7f739c927040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-18T13:03:04.262 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:04 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:04.105+0000 7f739c927040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-18T13:03:04.262 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:04 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:04.164+0000 7f739c927040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-18T13:03:04.313 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:03 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:03.925+0000 7f7f60f9e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-18T13:03:04.314 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:04 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:04.041+0000 7f7f60f9e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-18T13:03:04.314 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:04 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:04.099+0000 7f7f60f9e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-18T13:03:04.314 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:04 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:04.314+0000 7f7f60f9e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-18T13:03:04.525 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:04 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:04.386+0000 7f739c927040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-18T13:03:04.525 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:04 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:04.455+0000 7f739c927040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-18T13:03:04.526 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:04 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:04.524+0000 7f739c927040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-18T13:03:04.594 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:04 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:04.376+0000 7f7f60f9e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-18T13:03:04.594 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:04 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:04.445+0000 7f7f60f9e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-18T13:03:04.594 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:04 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:04.594+0000 7f7f60f9e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-18T13:03:04.794 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:04 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:04.675+0000 7f739c927040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-18T13:03:05.336 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:05 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:05.016+0000 7f7f60f9e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-18T13:03:05.429 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:05.122+0000 7f739c927040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-18T13:03:05.636 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:05 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:05.337+0000 7f7f60f9e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-18T13:03:05.636 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:05 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:05.405+0000 7f7f60f9e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-18T13:03:05.637 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:05 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:05.557+0000 7f7f60f9e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-18T13:03:05.694 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:05.462+0000 7f739c927040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-18T13:03:05.694 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:05.533+0000 7f739c927040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-18T13:03:05.695 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:05.693+0000 7f739c927040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-18T13:03:05.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.742+0000 7fcf83769640 1 -- 172.21.15.167:0/2723266857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf7c101e50 msgr2=0x7fcf7c102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:05.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.742+0000 7fcf83769640 1 --2- 172.21.15.167:0/2723266857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf7c101e50 0x7fcf7c102290 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009890 tx=0x7fcf6c02a380 comp rx=0 tx=0).stop 2024-06-18T13:03:05.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.742+0000 7fcf83769640 1 -- 172.21.15.167:0/2723266857 shutdown_connections 2024-06-18T13:03:05.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.742+0000 7fcf83769640 1 --2- 172.21.15.167:0/2723266857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf7c1027d0 0x7fcf7c10d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:05.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.742+0000 7fcf83769640 1 --2- 172.21.15.167:0/2723266857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf7c101e50 0x7fcf7c102290 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:05.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.742+0000 7fcf83769640 1 --2- 172.21.15.167:0/2723266857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf7c100560 0x7fcf7c100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:05.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.742+0000 7fcf83769640 1 -- 172.21.15.167:0/2723266857 >> 172.21.15.167:0/2723266857 conn(0x7fcf7c0fbf80 msgr2=0x7fcf7c0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:05.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.742+0000 7fcf83769640 1 -- 172.21.15.167:0/2723266857 shutdown_connections 2024-06-18T13:03:05.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.742+0000 7fcf83769640 1 -- 172.21.15.167:0/2723266857 wait complete. 2024-06-18T13:03:05.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.743+0000 7fcf83769640 1 Processor -- start 2024-06-18T13:03:05.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.743+0000 7fcf83769640 1 -- start start 2024-06-18T13:03:05.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf83769640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf7c100560 0x7fcf7c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:05.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf83769640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf7c101e50 0x7fcf7c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:05.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf83769640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf7c1027d0 0x7fcf7c19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:05.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf83769640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c10fb60 con 0x7fcf7c100560 2024-06-18T13:03:05.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf83769640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c10f9e0 con 0x7fcf7c101e50 2024-06-18T13:03:05.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf83769640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7c10fce0 con 0x7fcf7c1027d0 2024-06-18T13:03:05.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf81505640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf7c100560 0x7fcf7c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:05.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf81505640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf7c100560 0x7fcf7c1972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39592/0 (socket says 172.21.15.167:39592) 2024-06-18T13:03:05.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf81505640 1 -- 172.21.15.167:0/3478288057 learned_addr learned my addr 172.21.15.167:0/3478288057 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:05.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf80d04640 1 --2- 172.21.15.167:0/3478288057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf7c101e50 0x7fcf7c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:05.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf81d06640 1 --2- 172.21.15.167:0/3478288057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf7c1027d0 0x7fcf7c19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:05.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf81505640 1 -- 172.21.15.167:0/3478288057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf7c1027d0 msgr2=0x7fcf7c19bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:05.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf81505640 1 --2- 172.21.15.167:0/3478288057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf7c1027d0 0x7fcf7c19bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:05.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf81505640 1 -- 172.21.15.167:0/3478288057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf7c101e50 msgr2=0x7fcf7c1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:05.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf81505640 1 --2- 172.21.15.167:0/3478288057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf7c101e50 0x7fcf7c1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:05.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf81505640 1 -- 172.21.15.167:0/3478288057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c19c260 con 0x7fcf7c100560 2024-06-18T13:03:05.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf80d04640 1 --2- 172.21.15.167:0/3478288057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf7c101e50 0x7fcf7c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:03:05.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf81505640 1 --2- 172.21.15.167:0/3478288057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf7c100560 0x7fcf7c1972b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800e9f0 tx=0x7fcf7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:05.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf727fc640 1 -- 172.21.15.167:0/3478288057 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7800cd80 con 0x7fcf7c100560 2024-06-18T13:03:05.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf727fc640 1 -- 172.21.15.167:0/3478288057 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fcf780054a0 con 0x7fcf7c100560 2024-06-18T13:03:05.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.744+0000 7fcf83769640 1 -- 172.21.15.167:0/3478288057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7c19c550 con 0x7fcf7c100560 2024-06-18T13:03:05.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.745+0000 7fcf727fc640 1 -- 172.21.15.167:0/3478288057 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78010430 con 0x7fcf7c100560 2024-06-18T13:03:05.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.745+0000 7fcf83769640 1 -- 172.21.15.167:0/3478288057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7c1a3e30 con 0x7fcf7c100560 2024-06-18T13:03:05.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.746+0000 7fcf83769640 1 -- 172.21.15.167:0/3478288057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf7c100940 con 0x7fcf7c100560 2024-06-18T13:03:05.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.746+0000 7fcf727fc640 1 -- 172.21.15.167:0/3478288057 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 92182+0+0 (secure 0 0 0) 0x7fcf78004fe0 con 0x7fcf7c100560 2024-06-18T13:03:05.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.747+0000 7fcf727fc640 1 --2- 172.21.15.167:0/3478288057 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcf6806e030 0x7fcf680704f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:05.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.747+0000 7fcf80d04640 1 -- 172.21.15.167:0/3478288057 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcf6806e030 msgr2=0x7fcf680704f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.167:6800/1145517801 2024-06-18T13:03:05.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.747+0000 7fcf727fc640 1 -- 172.21.15.167:0/3478288057 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7148+0+0 (secure 0 0 0) 0x7fcf7808e890 con 0x7fcf7c100560 2024-06-18T13:03:05.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.747+0000 7fcf80d04640 1 --2- 172.21.15.167:0/3478288057 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcf6806e030 0x7fcf680704f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-18T13:03:05.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.751+0000 7fcf727fc640 1 -- 172.21.15.167:0/3478288057 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fcf7805b790 con 0x7fcf7c100560 2024-06-18T13:03:05.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.932+0000 7fcf83769640 1 -- 172.21.15.167:0/3478288057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf7c1a4000 con 0x7fcf7c100560 2024-06-18T13:03:05.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.934+0000 7fcf727fc640 1 -- 172.21.15.167:0/3478288057 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+22689 (secure 0 0 0) 0x7fcf7805fa90 con 0x7fcf7c100560 2024-06-18T13:03:05.935 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T13:03:05.935 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":70,"fsid":"3434f1e2-2d72-11ef-bc9d-c7b262605968","created":"2024-06-18T12:57:18.945902+0000","modified":"2024-06-18T13:02:43.605231+0000","last_up_change":"2024-06-18T13:02:14.621219+0000","last_in_change":"2024-06-18T13:01:57.489323+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-18T13:00:21.794480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-06-18T13:02:28.984540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-06-18T13:02:29.218378+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-06-18T13:02:30.723313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-06-18T13:02:32.668197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-06-18T13:02:34.735189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df01b396-984c-4e66-83a8-e439554c688a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3001811419},{"type":"v1","addr":"172.21.15.167:6803","nonce":3001811419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3001811419},{"type":"v1","addr":"172.21.15.167:6805","nonce":3001811419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3001811419},{"type":"v1","addr":"172.21.15.167:6809","nonce":3001811419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3001811419},{"type":"v1","addr":"172.21.15.167:6807","nonce":3001811419}]},"public_addr":"172.21.15.167:6803/3001811419","cluster_addr":"172.21.15.167:6805/3001811419","heartbeat_back_addr":"172.21.15.167:6809/3001811419","heartbeat_front_addr":"172.21.15.167:6807/3001811419","state":["exists","up"]},{"osd":1,"uuid":"d8159aff-9414-4ebe-897f-ffbf27ed9e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2010202972},{"type":"v1","addr":"172.21.15.167:6811","nonce":2010202972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2010202972},{"type":"v1","addr":"172.21.15.167:6813","nonce":2010202972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2010202972},{"type":"v1","addr":"172.21.15.167:6817","nonce":2010202972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2010202972},{"type":"v1","addr":"172.21.15.167:6815","nonce":2010202972}]},"public_addr":"172.21.15.167:6811/2010202972","cluster_addr":"172.21.15.167:6813/2010202972","heartbeat_back_addr":"172.21.15.167:6817/2010202972","heartbeat_front_addr":"172.21.15.167:6815/2010202972","state":["exists","up"]},{"osd":2,"uuid":"81482f44-3d16-4c10-b2ac-88eeeff554fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":183062921},{"type":"v1","addr":"172.21.15.167:6819","nonce":183062921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":183062921},{"type":"v1","addr":"172.21.15.167:6821","nonce":183062921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":183062921},{"type":"v1","addr":"172.21.15.167:6825","nonce":183062921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":183062921},{"type":"v1","addr":"172.21.15.167:6823","nonce":183062921}]},"public_addr":"172.21.15.167:6819/183062921","cluster_addr":"172.21.15.167:6821/183062921","heartbeat_back_addr":"172.21.15.167:6825/183062921","heartbeat_front_addr":"172.21.15.167:6823/183062921","state":["exists","up"]},{"osd":3,"uuid":"5958775e-faaa-4fba-9b33-dcd46c049926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":2828000547},{"type":"v1","addr":"172.21.15.167:6827","nonce":2828000547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":2828000547},{"type":"v1","addr":"172.21.15.167:6829","nonce":2828000547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828000547},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828000547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":2828000547},{"type":"v1","addr":"172.21.15.167:6831","nonce":2828000547}]},"public_addr":"172.21.15.167:6827/2828000547","cluster_addr":"172.21.15.167:6829/2828000547","heartbeat_back_addr":"172.21.15.167:6833/2828000547","heartbeat_front_addr":"172.21.15.167:6831/2828000547","state":["exists","up"]},{"osd":4,"uuid":"6abe43b9-eb57-4e7c-afce-fa7ff1b829c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3925771696},{"type":"v1","addr":"172.21.15.179:6801","nonce":3925771696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3925771696},{"type":"v1","addr":"172.21.15.179:6803","nonce":3925771696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3925771696},{"type":"v1","addr":"172.21.15.179:6807","nonce":3925771696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3925771696},{"type":"v1","addr":"172.21.15.179:6805","nonce":3925771696}]},"public_addr":"172.21.15.179:6801/3925771696","cluster_addr":"172.21.15.179:6803/3925771696","heartbeat_back_addr":"172.21.15.179:6807/3925771696","heartbeat_front_addr":"172.21.15.179:6805/3925771696","state":["exists","up"]},{"osd":5,"uuid":"45d6f34f-ce3b-4465-baa2-b49aec949910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1550696992},{"type":"v1","addr":"172.21.15.179:6809","nonce":1550696992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1550696992},{"type":"v1","addr":"172.21.15.179:6811","nonce":1550696992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1550696992},{"type":"v1","addr":"172.21.15.179:6815","nonce":1550696992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1550696992},{"type":"v1","addr":"172.21.15.179:6813","nonce":1550696992}]},"public_addr":"172.21.15.179:6809/1550696992","cluster_addr":"172.21.15.179:6811/1550696992","heartbeat_back_addr":"172.21.15.179:6815/1550696992","heartbeat_front_addr":"172.21.15.179:6813/1550696992","state":["exists","up"]},{"osd":6,"uuid":"06a47b6b-eac6-440b-9037-b55bfaf29e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":153587065},{"type":"v1","addr":"172.21.15.179:6817","nonce":153587065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":153587065},{"type":"v1","addr":"172.21.15.179:6819","nonce":153587065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":153587065},{"type":"v1","addr":"172.21.15.179:6823","nonce":153587065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":153587065},{"type":"v1","addr":"172.21.15.179:6821","nonce":153587065}]},"public_addr":"172.21.15.179:6817/153587065","cluster_addr":"172.21.15.179:6819/153587065","heartbeat_back_addr":"172.21.15.179:6823/153587065","heartbeat_front_addr":"172.21.15.179:6821/153587065","state":["exists","up"]},{"osd":7,"uuid":"717295cb-4786-41e4-9ac0-40fcea4b0375","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":3253527249},{"type":"v1","addr":"172.21.15.179:6825","nonce":3253527249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3253527249},{"type":"v1","addr":"172.21.15.179:6827","nonce":3253527249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":3253527249},{"type":"v1","addr":"172.21.15.179:6831","nonce":3253527249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":3253527249},{"type":"v1","addr":"172.21.15.179:6829","nonce":3253527249}]},"public_addr":"172.21.15.179:6825/3253527249","cluster_addr":"172.21.15.179:6827/3253527249","heartbeat_back_addr":"172.21.15.179:6831/3253527249","heartbeat_front_addr":"172.21.15.179:6829/3253527249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T12:59:31.344584+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T12:59:54.440221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:00:18.463935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:00:42.274566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:01:03.689366+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:01:26.918997+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:01:50.098743+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:02:12.629691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:0/2519223545":"2024-06-19T12:58:13.490367+0000","172.21.15.167:0/3211351643":"2024-06-19T12:58:13.490367+0000","172.21.15.167:6801/1873453520":"2024-06-19T12:58:13.490367+0000","172.21.15.167:6800/1873453520":"2024-06-19T12:58:13.490367+0000","172.21.15.167:0/3011148832":"2024-06-19T12:58:13.490367+0000","172.21.15.167:0/3606279536":"2024-06-19T12:57:51.125515+0000","172.21.15.167:0/999980396":"2024-06-19T12:57:51.125515+0000","172.21.15.167:6801/2756279078":"2024-06-19T12:57:51.125515+0000","172.21.15.167:0/1156304959":"2024-06-19T12:57:51.125515+0000","172.21.15.167:6800/2756279078":"2024-06-19T12:57:51.125515+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-18T13:03:05.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.936+0000 7fcf83769640 1 -- 172.21.15.167:0/3478288057 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcf6806e030 msgr2=0x7fcf680704f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-18T13:03:05.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.936+0000 7fcf83769640 1 --2- 172.21.15.167:0/3478288057 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcf6806e030 0x7fcf680704f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:05.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.936+0000 7fcf83769640 1 -- 172.21.15.167:0/3478288057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf7c100560 msgr2=0x7fcf7c1972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:05.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.936+0000 7fcf83769640 1 --2- 172.21.15.167:0/3478288057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf7c100560 0x7fcf7c1972b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800e9f0 tx=0x7fcf7800edf0 comp rx=0 tx=0).stop 2024-06-18T13:03:05.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.936+0000 7fcf83769640 1 -- 172.21.15.167:0/3478288057 shutdown_connections 2024-06-18T13:03:05.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.936+0000 7fcf83769640 1 --2- 172.21.15.167:0/3478288057 >> [v2:172.21.15.167:6800/1145517801,v1:172.21.15.167:6801/1145517801] conn(0x7fcf6806e030 0x7fcf680704f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:05.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.936+0000 7fcf83769640 1 --2- 172.21.15.167:0/3478288057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf7c1027d0 0x7fcf7c19bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:05.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.936+0000 7fcf83769640 1 --2- 172.21.15.167:0/3478288057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf7c101e50 0x7fcf7c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:05.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.936+0000 7fcf83769640 1 --2- 172.21.15.167:0/3478288057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf7c100560 0x7fcf7c1972b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:05.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.936+0000 7fcf83769640 1 -- 172.21.15.167:0/3478288057 >> 172.21.15.167:0/3478288057 conn(0x7fcf7c0fbf80 msgr2=0x7fcf7c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:05.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.936+0000 7fcf83769640 1 -- 172.21.15.167:0/3478288057 shutdown_connections 2024-06-18T13:03:05.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:05.936+0000 7fcf83769640 1 -- 172.21.15.167:0/3478288057 wait complete. 2024-06-18T13:03:06.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3478288057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-18T13:03:06.137 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:05 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:05.871+0000 7f7f60f9e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-18T13:03:06.137 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:05 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:05.933+0000 7f7f60f9e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-18T13:03:06.137 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:06 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:06.120+0000 7f7f60f9e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-18T13:03:06.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3478288057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-18T13:03:06.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3478288057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-18T13:03:06.227 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:06 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:06.030+0000 7f739c927040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-18T13:03:06.227 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:06 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:06.093+0000 7f739c927040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-18T13:03:06.592 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:06 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:06.284+0000 7f739c927040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-18T13:03:06.592 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:06 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:06.436+0000 7f739c927040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-18T13:03:06.592 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:06 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:06.504+0000 7f739c927040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-18T13:03:06.592 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:06 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:06.590+0000 7f739c927040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-18T13:03:06.636 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:06 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:06.268+0000 7f7f60f9e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-18T13:03:06.636 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:06 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:06.332+0000 7f7f60f9e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-18T13:03:06.636 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:06 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:06.416+0000 7f7f60f9e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-18T13:03:06.636 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:06 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:06.479+0000 7f7f60f9e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-18T13:03:06.702 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-18T13:03:06.702 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd dump --format=json 2024-06-18T13:03:06.944 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:06.976 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:06 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:06.652+0000 7f739c927040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-18T13:03:07.288 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:07 smithi179 ceph-mon[33312]: Standby manager daemon x restarted 2024-06-18T13:03:07.288 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:07 smithi179 ceph-mon[33312]: Standby manager daemon x started 2024-06-18T13:03:07.288 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:07 smithi179 ceph-mon[33312]: from='mgr.? 172.21.15.179:0/3736650641' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-18T13:03:07.288 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:07 smithi179 ceph-mon[33312]: from='mgr.? 172.21.15.179:0/3736650641' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-18T13:03:07.288 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:07 smithi179 ceph-mon[33312]: from='mgr.? 172.21.15.179:0/3736650641' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-18T13:03:07.288 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:07 smithi179 ceph-mon[33312]: from='mgr.? 172.21.15.179:0/3736650641' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-18T13:03:07.289 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:07 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:07.020+0000 7f7f60f9e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-18T13:03:07.289 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:07 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:07.181+0000 7f7f60f9e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-18T13:03:07.289 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:07 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: [18/Jun/2024:13:03:07] ENGINE Bus STARTING 2024-06-18T13:03:07.289 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:07 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: CherryPy Checker: 2024-06-18T13:03:07.289 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:07 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: The Application mounted at '' has an empty config. 2024-06-18T13:03:07.289 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:07 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: 2024-06-18T13:03:07.289 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:07 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: [18/Jun/2024:13:03:07] ENGINE Serving on http://:::9283 2024-06-18T13:03:07.289 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:03:07 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x[35146]: [18/Jun/2024:13:03:07] ENGINE Bus STARTED 2024-06-18T13:03:07.319 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:07.233+0000 7f739c927040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-18T13:03:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-mon[37701]: Standby manager daemon x restarted 2024-06-18T13:03:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-mon[37701]: Standby manager daemon x started 2024-06-18T13:03:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-mon[37701]: from='mgr.? 172.21.15.179:0/3736650641' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-18T13:03:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-mon[37701]: from='mgr.? 172.21.15.179:0/3736650641' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-18T13:03:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-mon[37701]: from='mgr.? 172.21.15.179:0/3736650641' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-18T13:03:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-mon[37701]: from='mgr.? 172.21.15.179:0/3736650641' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-18T13:03:07.320 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-mon[29496]: Standby manager daemon x restarted 2024-06-18T13:03:07.320 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-mon[29496]: Standby manager daemon x started 2024-06-18T13:03:07.320 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-mon[29496]: from='mgr.? 172.21.15.179:0/3736650641' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-18T13:03:07.320 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-mon[29496]: from='mgr.? 172.21.15.179:0/3736650641' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-18T13:03:07.321 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-mon[29496]: from='mgr.? 172.21.15.179:0/3736650641' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-18T13:03:07.321 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-mon[29496]: from='mgr.? 172.21.15.179:0/3736650641' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-18T13:03:07.610 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:07.406+0000 7f739c927040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-18T13:03:07.610 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:03:07] ENGINE Bus STARTING 2024-06-18T13:03:07.610 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: CherryPy Checker: 2024-06-18T13:03:07.610 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: The Application mounted at '' has an empty config. 2024-06-18T13:03:07.610 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:07.892 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:03:07] ENGINE Serving on http://:::9283 2024-06-18T13:03:07.892 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:07 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:03:07] ENGINE Bus STARTED 2024-06-18T13:03:08.424 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: mgrmap e18: y(active, since 4m), standbys: x 2024-06-18T13:03:08.424 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: Active manager daemon y restarted 2024-06-18T13:03:08.424 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: Activating manager daemon y 2024-06-18T13:03:08.424 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: osdmap e71: 8 total, 8 up, 8 in 2024-06-18T13:03:08.424 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: mgrmap e19: y(active, starting, since 0.0125944s), standbys: x 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-18T13:03:08.425 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-18T13:03:08.426 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-18T13:03:08.426 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: Manager daemon y is now available 2024-06-18T13:03:08.426 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:03:08.426 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:08.426 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:03:08.426 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-18T13:03:08.426 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-18T13:03:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: mgrmap e18: y(active, since 4m), standbys: x 2024-06-18T13:03:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: Active manager daemon y restarted 2024-06-18T13:03:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: Activating manager daemon y 2024-06-18T13:03:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: osdmap e71: 8 total, 8 up, 8 in 2024-06-18T13:03:08.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: mgrmap e19: y(active, starting, since 0.0125944s), standbys: x 2024-06-18T13:03:08.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T13:03:08.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T13:03:08.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T13:03:08.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-18T13:03:08.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-18T13:03:08.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T13:03:08.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T13:03:08.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:03:08.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:03:08.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:03:08.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:03:08.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:03:08.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:03:08.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-18T13:03:08.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-18T13:03:08.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-18T13:03:08.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: Manager daemon y is now available 2024-06-18T13:03:08.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:03:08.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:08.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:03:08.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-18T13:03:08.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-18T13:03:08.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:03:08.603 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: mgrmap e18: y(active, since 4m), standbys: x 2024-06-18T13:03:08.603 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: Active manager daemon y restarted 2024-06-18T13:03:08.603 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: Activating manager daemon y 2024-06-18T13:03:08.603 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: osdmap e71: 8 total, 8 up, 8 in 2024-06-18T13:03:08.603 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: mgrmap e19: y(active, starting, since 0.0125944s), standbys: x 2024-06-18T13:03:08.603 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-18T13:03:08.603 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-18T13:03:08.603 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-18T13:03:08.603 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-18T13:03:08.603 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-18T13:03:08.603 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-18T13:03:08.603 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: Manager daemon y is now available 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-18T13:03:08.604 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-18T13:03:08.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.657+0000 7fc92e6de640 1 -- 172.21.15.167:0/2111812733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc928100560 msgr2=0x7fc928100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:08.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.657+0000 7fc92e6de640 1 --2- 172.21.15.167:0/2111812733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc928100560 0x7fc928100940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc918009930 tx=0x7fc91802a420 comp rx=0 tx=0).stop 2024-06-18T13:03:08.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 -- 172.21.15.167:0/2111812733 shutdown_connections 2024-06-18T13:03:08.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 --2- 172.21.15.167:0/2111812733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9281027d0 0x7fc92810d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:08.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 --2- 172.21.15.167:0/2111812733 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc928101e50 0x7fc928102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:08.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 --2- 172.21.15.167:0/2111812733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc928100560 0x7fc928100940 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 -- 172.21.15.167:0/2111812733 >> 172.21.15.167:0/2111812733 conn(0x7fc9280fbf80 msgr2=0x7fc9280fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 -- 172.21.15.167:0/2111812733 shutdown_connections 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 -- 172.21.15.167:0/2111812733 wait complete. 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 Processor -- start 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 -- start start 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc928100560 0x7fc928197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc928101e50 0x7fc928197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9281027d0 0x7fc92819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc92810fb90 con 0x7fc928100560 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc92810fa10 con 0x7fc928101e50 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.658+0000 7fc92e6de640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc92810fd10 con 0x7fc9281027d0 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc927fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc928100560 0x7fc928197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc92cc7b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9281027d0 0x7fc92819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc927fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc928100560 0x7fc928197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:39680/0 (socket says 172.21.15.167:39680) 2024-06-18T13:03:08.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc927fff640 1 -- 172.21.15.167:0/2148357128 learned_addr learned my addr 172.21.15.167:0/2148357128 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:08.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc91ffff640 1 --2- 172.21.15.167:0/2148357128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc928101e50 0x7fc928197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:08.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc92cc7b640 1 -- 172.21.15.167:0/2148357128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc928101e50 msgr2=0x7fc928197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:08.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc92cc7b640 1 --2- 172.21.15.167:0/2148357128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc928101e50 0x7fc928197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:08.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc92cc7b640 1 -- 172.21.15.167:0/2148357128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc928100560 msgr2=0x7fc928197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:08.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc92cc7b640 1 --2- 172.21.15.167:0/2148357128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc928100560 0x7fc928197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:08.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc92cc7b640 1 -- 172.21.15.167:0/2148357128 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92819c1c0 con 0x7fc9281027d0 2024-06-18T13:03:08.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc92cc7b640 1 --2- 172.21.15.167:0/2148357128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9281027d0 0x7fc92819bae0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc920009fd0 tx=0x7fc92000b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:08.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc925ffb640 1 -- 172.21.15.167:0/2148357128 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc920004b20 con 0x7fc9281027d0 2024-06-18T13:03:08.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc92e6de640 1 -- 172.21.15.167:0/2148357128 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92819c4b0 con 0x7fc9281027d0 2024-06-18T13:03:08.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc925ffb640 1 -- 172.21.15.167:0/2148357128 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fc920004cc0 con 0x7fc9281027d0 2024-06-18T13:03:08.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc925ffb640 1 -- 172.21.15.167:0/2148357128 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9200052c0 con 0x7fc9281027d0 2024-06-18T13:03:08.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.659+0000 7fc92e6de640 1 -- 172.21.15.167:0/2148357128 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9280fc360 con 0x7fc9281027d0 2024-06-18T13:03:08.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.660+0000 7fc92e6de640 1 -- 172.21.15.167:0/2148357128 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc928100940 con 0x7fc9281027d0 2024-06-18T13:03:08.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.661+0000 7fc925ffb640 1 -- 172.21.15.167:0/2148357128 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 20) ==== 91925+0+0 (secure 0 0 0) 0x7fc9200054e0 con 0x7fc9281027d0 2024-06-18T13:03:08.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.662+0000 7fc925ffb640 1 --2- 172.21.15.167:0/2148357128 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fc91406dd40 0x7fc914070200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:08.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.662+0000 7fc925ffb640 1 -- 172.21.15.167:0/2148357128 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7fc920092010 con 0x7fc9281027d0 2024-06-18T13:03:08.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.662+0000 7fc927fff640 1 --2- 172.21.15.167:0/2148357128 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fc91406dd40 0x7fc914070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:08.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.662+0000 7fc927fff640 1 --2- 172.21.15.167:0/2148357128 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fc91406dd40 0x7fc914070200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc918000c00 tx=0x7fc918005a20 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:08.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.666+0000 7fc925ffb640 1 -- 172.21.15.167:0/2148357128 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fc92005f070 con 0x7fc9281027d0 2024-06-18T13:03:08.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.851+0000 7fc92e6de640 1 -- 172.21.15.167:0/2148357128 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9280fc680 con 0x7fc9281027d0 2024-06-18T13:03:08.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.853+0000 7fc925ffb640 1 -- 172.21.15.167:0/2148357128 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+23090 (secure 0 0 0) 0x7fc920063370 con 0x7fc9281027d0 2024-06-18T13:03:08.854 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T13:03:08.854 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":71,"fsid":"3434f1e2-2d72-11ef-bc9d-c7b262605968","created":"2024-06-18T12:57:18.945902+0000","modified":"2024-06-18T13:03:07.410775+0000","last_up_change":"2024-06-18T13:02:14.621219+0000","last_in_change":"2024-06-18T13:01:57.489323+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-18T13:00:21.794480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-06-18T13:02:28.984540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-06-18T13:02:29.218378+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-06-18T13:02:30.723313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-06-18T13:02:32.668197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-06-18T13:02:34.735189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df01b396-984c-4e66-83a8-e439554c688a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3001811419},{"type":"v1","addr":"172.21.15.167:6803","nonce":3001811419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3001811419},{"type":"v1","addr":"172.21.15.167:6805","nonce":3001811419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3001811419},{"type":"v1","addr":"172.21.15.167:6809","nonce":3001811419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3001811419},{"type":"v1","addr":"172.21.15.167:6807","nonce":3001811419}]},"public_addr":"172.21.15.167:6803/3001811419","cluster_addr":"172.21.15.167:6805/3001811419","heartbeat_back_addr":"172.21.15.167:6809/3001811419","heartbeat_front_addr":"172.21.15.167:6807/3001811419","state":["exists","up"]},{"osd":1,"uuid":"d8159aff-9414-4ebe-897f-ffbf27ed9e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2010202972},{"type":"v1","addr":"172.21.15.167:6811","nonce":2010202972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2010202972},{"type":"v1","addr":"172.21.15.167:6813","nonce":2010202972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2010202972},{"type":"v1","addr":"172.21.15.167:6817","nonce":2010202972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2010202972},{"type":"v1","addr":"172.21.15.167:6815","nonce":2010202972}]},"public_addr":"172.21.15.167:6811/2010202972","cluster_addr":"172.21.15.167:6813/2010202972","heartbeat_back_addr":"172.21.15.167:6817/2010202972","heartbeat_front_addr":"172.21.15.167:6815/2010202972","state":["exists","up"]},{"osd":2,"uuid":"81482f44-3d16-4c10-b2ac-88eeeff554fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":183062921},{"type":"v1","addr":"172.21.15.167:6819","nonce":183062921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":183062921},{"type":"v1","addr":"172.21.15.167:6821","nonce":183062921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":183062921},{"type":"v1","addr":"172.21.15.167:6825","nonce":183062921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":183062921},{"type":"v1","addr":"172.21.15.167:6823","nonce":183062921}]},"public_addr":"172.21.15.167:6819/183062921","cluster_addr":"172.21.15.167:6821/183062921","heartbeat_back_addr":"172.21.15.167:6825/183062921","heartbeat_front_addr":"172.21.15.167:6823/183062921","state":["exists","up"]},{"osd":3,"uuid":"5958775e-faaa-4fba-9b33-dcd46c049926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":2828000547},{"type":"v1","addr":"172.21.15.167:6827","nonce":2828000547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":2828000547},{"type":"v1","addr":"172.21.15.167:6829","nonce":2828000547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828000547},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828000547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":2828000547},{"type":"v1","addr":"172.21.15.167:6831","nonce":2828000547}]},"public_addr":"172.21.15.167:6827/2828000547","cluster_addr":"172.21.15.167:6829/2828000547","heartbeat_back_addr":"172.21.15.167:6833/2828000547","heartbeat_front_addr":"172.21.15.167:6831/2828000547","state":["exists","up"]},{"osd":4,"uuid":"6abe43b9-eb57-4e7c-afce-fa7ff1b829c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3925771696},{"type":"v1","addr":"172.21.15.179:6801","nonce":3925771696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3925771696},{"type":"v1","addr":"172.21.15.179:6803","nonce":3925771696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3925771696},{"type":"v1","addr":"172.21.15.179:6807","nonce":3925771696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3925771696},{"type":"v1","addr":"172.21.15.179:6805","nonce":3925771696}]},"public_addr":"172.21.15.179:6801/3925771696","cluster_addr":"172.21.15.179:6803/3925771696","heartbeat_back_addr":"172.21.15.179:6807/3925771696","heartbeat_front_addr":"172.21.15.179:6805/3925771696","state":["exists","up"]},{"osd":5,"uuid":"45d6f34f-ce3b-4465-baa2-b49aec949910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1550696992},{"type":"v1","addr":"172.21.15.179:6809","nonce":1550696992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1550696992},{"type":"v1","addr":"172.21.15.179:6811","nonce":1550696992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1550696992},{"type":"v1","addr":"172.21.15.179:6815","nonce":1550696992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1550696992},{"type":"v1","addr":"172.21.15.179:6813","nonce":1550696992}]},"public_addr":"172.21.15.179:6809/1550696992","cluster_addr":"172.21.15.179:6811/1550696992","heartbeat_back_addr":"172.21.15.179:6815/1550696992","heartbeat_front_addr":"172.21.15.179:6813/1550696992","state":["exists","up"]},{"osd":6,"uuid":"06a47b6b-eac6-440b-9037-b55bfaf29e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":153587065},{"type":"v1","addr":"172.21.15.179:6817","nonce":153587065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":153587065},{"type":"v1","addr":"172.21.15.179:6819","nonce":153587065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":153587065},{"type":"v1","addr":"172.21.15.179:6823","nonce":153587065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":153587065},{"type":"v1","addr":"172.21.15.179:6821","nonce":153587065}]},"public_addr":"172.21.15.179:6817/153587065","cluster_addr":"172.21.15.179:6819/153587065","heartbeat_back_addr":"172.21.15.179:6823/153587065","heartbeat_front_addr":"172.21.15.179:6821/153587065","state":["exists","up"]},{"osd":7,"uuid":"717295cb-4786-41e4-9ac0-40fcea4b0375","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":3253527249},{"type":"v1","addr":"172.21.15.179:6825","nonce":3253527249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3253527249},{"type":"v1","addr":"172.21.15.179:6827","nonce":3253527249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":3253527249},{"type":"v1","addr":"172.21.15.179:6831","nonce":3253527249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":3253527249},{"type":"v1","addr":"172.21.15.179:6829","nonce":3253527249}]},"public_addr":"172.21.15.179:6825/3253527249","cluster_addr":"172.21.15.179:6827/3253527249","heartbeat_back_addr":"172.21.15.179:6831/3253527249","heartbeat_front_addr":"172.21.15.179:6829/3253527249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T12:59:31.344584+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T12:59:54.440221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:00:18.463935+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:00:42.274566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:01:03.689366+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:01:26.918997+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:01:50.098743+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-18T13:02:12.629691+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6801/1145517801":"2024-06-19T13:03:07.410729+0000","172.21.15.167:0/1266240186":"2024-06-19T13:03:07.410729+0000","172.21.15.167:6800/1145517801":"2024-06-19T13:03:07.410729+0000","172.21.15.167:0/506888618":"2024-06-19T13:03:07.410729+0000","172.21.15.167:6801/2756279078":"2024-06-19T12:57:51.125515+0000","172.21.15.167:0/999980396":"2024-06-19T12:57:51.125515+0000","172.21.15.167:0/134973485":"2024-06-19T13:03:07.410729+0000","172.21.15.167:0/3011148832":"2024-06-19T12:58:13.490367+0000","172.21.15.167:6800/2756279078":"2024-06-19T12:57:51.125515+0000","172.21.15.167:6800/1873453520":"2024-06-19T12:58:13.490367+0000","172.21.15.167:6801/1873453520":"2024-06-19T12:58:13.490367+0000","172.21.15.167:0/3211351643":"2024-06-19T12:58:13.490367+0000","172.21.15.167:0/3606279536":"2024-06-19T12:57:51.125515+0000","172.21.15.167:0/499306102":"2024-06-19T13:03:07.410729+0000","172.21.15.167:0/2519223545":"2024-06-19T12:58:13.490367+0000","172.21.15.167:0/1156304959":"2024-06-19T12:57:51.125515+0000","172.21.15.167:0/3362690351":"2024-06-19T13:03:07.410729+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-18T13:03:08.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.855+0000 7fc92e6de640 1 -- 172.21.15.167:0/2148357128 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fc91406dd40 msgr2=0x7fc914070200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:08.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.855+0000 7fc92e6de640 1 --2- 172.21.15.167:0/2148357128 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fc91406dd40 0x7fc914070200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc918000c00 tx=0x7fc918005a20 comp rx=0 tx=0).stop 2024-06-18T13:03:08.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.855+0000 7fc92e6de640 1 -- 172.21.15.167:0/2148357128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9281027d0 msgr2=0x7fc92819bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:08.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.855+0000 7fc92e6de640 1 --2- 172.21.15.167:0/2148357128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9281027d0 0x7fc92819bae0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc920009fd0 tx=0x7fc92000b690 comp rx=0 tx=0).stop 2024-06-18T13:03:08.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.855+0000 7fc92e6de640 1 -- 172.21.15.167:0/2148357128 shutdown_connections 2024-06-18T13:03:08.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.855+0000 7fc92e6de640 1 --2- 172.21.15.167:0/2148357128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9281027d0 0x7fc92819bae0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:08.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.855+0000 7fc92e6de640 1 --2- 172.21.15.167:0/2148357128 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fc91406dd40 0x7fc914070200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:08.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.855+0000 7fc92e6de640 1 --2- 172.21.15.167:0/2148357128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc928101e50 0x7fc928197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:08.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.855+0000 7fc92e6de640 1 --2- 172.21.15.167:0/2148357128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc928100560 0x7fc928197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:08.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.855+0000 7fc92e6de640 1 -- 172.21.15.167:0/2148357128 >> 172.21.15.167:0/2148357128 conn(0x7fc9280fbf80 msgr2=0x7fc928102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:08.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.856+0000 7fc92e6de640 1 -- 172.21.15.167:0/2148357128 shutdown_connections 2024-06-18T13:03:08.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:08.856+0000 7fc92e6de640 1 -- 172.21.15.167:0/2148357128 wait complete. 2024-06-18T13:03:09.231 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:09.137+0000 7f72f43d8640 -1 log_channel(cephadm) log [ERR] : [18/Jun/2024:13:03:09] ENGINE Error in HTTPServer.serve 2024-06-18T13:03:09.232 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: Traceback (most recent call last): 2024-06-18T13:03:09.232 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-18T13:03:09.232 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: self._connections.run(self.expiration_interval) 2024-06-18T13:03:09.232 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-18T13:03:09.232 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: self._run(expiration_interval) 2024-06-18T13:03:09.232 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-18T13:03:09.232 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: new_conn = self._from_server_socket(self.server.socket) 2024-06-18T13:03:09.232 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-18T13:03:09.232 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-18T13:03:09.232 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-18T13:03:09.232 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: s = self.context.wrap_socket( 2024-06-18T13:03:09.232 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-18T13:03:09.232 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: return self.sslsocket_class._create( 2024-06-18T13:03:09.233 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-18T13:03:09.233 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: self.do_handshake() 2024-06-18T13:03:09.233 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-18T13:03:09.233 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: self._sslobj.do_handshake() 2024-06-18T13:03:09.233 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-18T13:03:09.233 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:03:09.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-mon[37701]: mgrmap e20: y(active, since 1.01753s), standbys: x 2024-06-18T13:03:09.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2148357128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-18T13:03:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-mon[37701]: [18/Jun/2024:13:03:08] ENGINE Bus STARTING 2024-06-18T13:03:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-mon[37701]: [18/Jun/2024:13:03:09] ENGINE Serving on http://172.21.15.167:8765 2024-06-18T13:03:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-mon[37701]: [18/Jun/2024:13:03:09] ENGINE Serving on https://172.21.15.167:7150 2024-06-18T13:03:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-mon[37701]: [18/Jun/2024:13:03:09] ENGINE Bus STARTED 2024-06-18T13:03:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-mon[37701]: [18/Jun/2024:13:03:09] ENGINE Error in HTTPServer.serve 2024-06-18T13:03:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout: Traceback (most recent call last): 2024-06-18T13:03:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-18T13:03:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout: self._connections.run(self.expiration_interval) 2024-06-18T13:03:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-18T13:03:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout: self._run(expiration_interval) 2024-06-18T13:03:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-18T13:03:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-18T13:03:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-18T13:03:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-18T13:03:09.592 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-18T13:03:09.592 INFO:journalctl@ceph.mon.c.smithi167.stdout: s = self.context.wrap_socket( 2024-06-18T13:03:09.592 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-18T13:03:09.592 INFO:journalctl@ceph.mon.c.smithi167.stdout: return self.sslsocket_class._create( 2024-06-18T13:03:09.592 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-18T13:03:09.592 INFO:journalctl@ceph.mon.c.smithi167.stdout: self.do_handshake() 2024-06-18T13:03:09.592 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-18T13:03:09.592 INFO:journalctl@ceph.mon.c.smithi167.stdout: self._sslobj.do_handshake() 2024-06-18T13:03:09.592 INFO:journalctl@ceph.mon.c.smithi167.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-18T13:03:09.592 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-mon[29496]: mgrmap e20: y(active, since 1.01753s), standbys: x 2024-06-18T13:03:09.592 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2148357128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-18T13:03:09.592 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-mon[29496]: [18/Jun/2024:13:03:08] ENGINE Bus STARTING 2024-06-18T13:03:09.592 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-mon[29496]: [18/Jun/2024:13:03:09] ENGINE Serving on http://172.21.15.167:8765 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-mon[29496]: [18/Jun/2024:13:03:09] ENGINE Serving on https://172.21.15.167:7150 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-mon[29496]: [18/Jun/2024:13:03:09] ENGINE Bus STARTED 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:09 smithi167 ceph-mon[29496]: [18/Jun/2024:13:03:09] ENGINE Error in HTTPServer.serve 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout: Traceback (most recent call last): 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout: self._connections.run(self.expiration_interval) 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout: self._run(expiration_interval) 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout: s = self.context.wrap_socket( 2024-06-18T13:03:09.593 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-18T13:03:09.594 INFO:journalctl@ceph.mon.a.smithi167.stdout: return self.sslsocket_class._create( 2024-06-18T13:03:09.594 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-18T13:03:09.594 INFO:journalctl@ceph.mon.a.smithi167.stdout: self.do_handshake() 2024-06-18T13:03:09.594 INFO:journalctl@ceph.mon.a.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-18T13:03:09.594 INFO:journalctl@ceph.mon.a.smithi167.stdout: self._sslobj.do_handshake() 2024-06-18T13:03:09.594 INFO:journalctl@ceph.mon.a.smithi167.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-18T13:03:09.616 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-18T13:03:09.617 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-18T13:03:09.617 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-18T13:03:09.617 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-18T13:03:09.618 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-18T13:03:09.618 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-18T13:03:09.618 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-18T13:03:09.619 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-18T13:03:09.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:09 smithi179 ceph-mon[33312]: mgrmap e20: y(active, since 1.01753s), standbys: x 2024-06-18T13:03:09.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2148357128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:09 smithi179 ceph-mon[33312]: [18/Jun/2024:13:03:08] ENGINE Bus STARTING 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:09 smithi179 ceph-mon[33312]: [18/Jun/2024:13:03:09] ENGINE Serving on http://172.21.15.167:8765 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:09 smithi179 ceph-mon[33312]: [18/Jun/2024:13:03:09] ENGINE Serving on https://172.21.15.167:7150 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:09 smithi179 ceph-mon[33312]: [18/Jun/2024:13:03:09] ENGINE Bus STARTED 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:09 smithi179 ceph-mon[33312]: [18/Jun/2024:13:03:09] ENGINE Error in HTTPServer.serve 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout: Traceback (most recent call last): 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout: self._connections.run(self.expiration_interval) 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout: self._run(expiration_interval) 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-18T13:03:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout: s = self.context.wrap_socket( 2024-06-18T13:03:09.638 INFO:journalctl@ceph.mon.b.smithi179.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-18T13:03:09.638 INFO:journalctl@ceph.mon.b.smithi179.stdout: return self.sslsocket_class._create( 2024-06-18T13:03:09.638 INFO:journalctl@ceph.mon.b.smithi179.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-18T13:03:09.638 INFO:journalctl@ceph.mon.b.smithi179.stdout: self.do_handshake() 2024-06-18T13:03:09.638 INFO:journalctl@ceph.mon.b.smithi179.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-18T13:03:09.638 INFO:journalctl@ceph.mon.b.smithi179.stdout: self._sslobj.do_handshake() 2024-06-18T13:03:09.638 INFO:journalctl@ceph.mon.b.smithi179.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-18T13:03:10.138 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:10.184 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:10.187 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:10.187 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:10.189 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:10.200 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:10.207 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:10.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:10 smithi167 ceph-mon[37701]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:03:10.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:10 smithi167 ceph-mon[37701]: mgrmap e21: y(active, since 2s), standbys: x 2024-06-18T13:03:10.554 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:10 smithi167 ceph-mon[29496]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:03:10.554 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:10 smithi167 ceph-mon[29496]: mgrmap e21: y(active, since 2s), standbys: x 2024-06-18T13:03:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:10 smithi179 ceph-mon[33312]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:03:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:10 smithi179 ceph-mon[33312]: mgrmap e21: y(active, since 2s), standbys: x 2024-06-18T13:03:11.875 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:11.875 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:11.875 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:11.875 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:11.875 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:11.875 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:11.875 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:11.875 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:11.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[37701]: Adjusting osd_memory_target on smithi179 to 3760M 2024-06-18T13:03:11.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:11.876 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:11.876 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:11.876 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:11.876 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:11.876 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:11.876 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:11.876 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:11.876 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:11.876 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[29496]: Adjusting osd_memory_target on smithi179 to 3760M 2024-06-18T13:03:11.877 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:11.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:11.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:11.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:11 smithi179 ceph-mon[33312]: Adjusting osd_memory_target on smithi179 to 3760M 2024-06-18T13:03:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:13.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:13 smithi167 ceph-mon[37701]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:03:13.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:13 smithi167 ceph-mon[37701]: mgrmap e22: y(active, since 4s), standbys: x 2024-06-18T13:03:13.371 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:13 smithi167 ceph-mon[29496]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:03:13.371 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:13 smithi167 ceph-mon[29496]: mgrmap e22: y(active, since 4s), standbys: x 2024-06-18T13:03:13.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:13 smithi179 ceph-mon[33312]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:03:13.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:13 smithi179 ceph-mon[33312]: mgrmap e22: y(active, since 4s), standbys: x 2024-06-18T13:03:15.344 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:15 smithi167 ceph-mon[37701]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:03:15.345 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:03:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:03:15.345 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:15 smithi167 ceph-mon[29496]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:03:15.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:15 smithi179 ceph-mon[33312]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:03:17.403 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:17 smithi167 ceph-mon[37701]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-18T13:03:17.403 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:17 smithi167 ceph-mon[29496]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-18T13:03:17.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.575+0000 7f239cffc640 1 -- 172.21.15.167:0/3789758551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2398101f40 msgr2=0x7f2398105f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:17.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.575+0000 7f239cffc640 1 --2- 172.21.15.167:0/3789758551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2398101f40 0x7f2398105f90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2380009900 tx=0x7f238002a410 comp rx=0 tx=0).stop 2024-06-18T13:03:17.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.575+0000 7f239cffc640 1 -- 172.21.15.167:0/3789758551 shutdown_connections 2024-06-18T13:03:17.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.575+0000 7f239cffc640 1 --2- 172.21.15.167:0/3789758551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23981066c0 0x7f239810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:17.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.575+0000 7f239cffc640 1 --2- 172.21.15.167:0/3789758551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2398101f40 0x7f2398105f90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:17.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.575+0000 7f239cffc640 1 --2- 172.21.15.167:0/3789758551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2398101590 0x7f2398101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:17.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.575+0000 7f239cffc640 1 -- 172.21.15.167:0/3789758551 >> 172.21.15.167:0/3789758551 conn(0x7f23980fd070 msgr2=0x7f23980ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:17.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.575+0000 7f239cffc640 1 -- 172.21.15.167:0/3789758551 shutdown_connections 2024-06-18T13:03:17.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.575+0000 7f239cffc640 1 -- 172.21.15.167:0/3789758551 wait complete. 2024-06-18T13:03:17.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.576+0000 7f239cffc640 1 Processor -- start 2024-06-18T13:03:17.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.576+0000 7f239cffc640 1 -- start start 2024-06-18T13:03:17.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.576+0000 7f239cffc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2398101590 0x7f23981971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:17.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.576+0000 7f239cffc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2398101f40 0x7f2398197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:17.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.576+0000 7f239cffc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23981066c0 0x7f239819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:17.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.576+0000 7f239cffc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2398110bf0 con 0x7f2398101590 2024-06-18T13:03:17.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.576+0000 7f239cffc640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2398110a70 con 0x7f23981066c0 2024-06-18T13:03:17.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.576+0000 7f239cffc640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2398110d70 con 0x7f2398101f40 2024-06-18T13:03:17.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.576+0000 7f239659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2398101590 0x7f23981971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:17.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.576+0000 7f238dd9b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2398101f40 0x7f2398197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:17.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.576+0000 7f239659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2398101590 0x7f23981971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:50096/0 (socket says 172.21.15.167:50096) 2024-06-18T13:03:17.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.576+0000 7f239659c640 1 -- 172.21.15.167:0/3560696935 learned_addr learned my addr 172.21.15.167:0/3560696935 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.577+0000 7f239659c640 1 -- 172.21.15.167:0/3560696935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2398101f40 msgr2=0x7f2398197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.577+0000 7f239659c640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2398101f40 0x7f2398197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.577+0000 7f2396d9d640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23981066c0 0x7f239819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.577+0000 7f239659c640 1 -- 172.21.15.167:0/3560696935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23981066c0 msgr2=0x7f239819bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.577+0000 7f239659c640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23981066c0 0x7f239819bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.577+0000 7f239659c640 1 -- 172.21.15.167:0/3560696935 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239819c1a0 con 0x7f2398101590 2024-06-18T13:03:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.577+0000 7f238dd9b640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2398101f40 0x7f2398197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:03:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.577+0000 7f2396d9d640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23981066c0 0x7f239819bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-18T13:03:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.577+0000 7f239659c640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2398101590 0x7f23981971f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f238800ea60 tx=0x7f238800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.577+0000 7f238ffff640 1 -- 172.21.15.167:0/3560696935 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238800cde0 con 0x7f2398101590 2024-06-18T13:03:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.577+0000 7f239cffc640 1 -- 172.21.15.167:0/3560696935 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239819c490 con 0x7f2398101590 2024-06-18T13:03:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.577+0000 7f238ffff640 1 -- 172.21.15.167:0/3560696935 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f23880035f0 con 0x7f2398101590 2024-06-18T13:03:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.577+0000 7f238ffff640 1 -- 172.21.15.167:0/3560696935 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23880044a0 con 0x7f2398101590 2024-06-18T13:03:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.577+0000 7f239cffc640 1 -- 172.21.15.167:0/3560696935 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23981a3d70 con 0x7f2398101590 2024-06-18T13:03:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.578+0000 7f239cffc640 1 -- 172.21.15.167:0/3560696935 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f23980fe0d0 con 0x7f2398101590 2024-06-18T13:03:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.578+0000 7f238ffff640 1 -- 172.21.15.167:0/3560696935 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f2388004020 con 0x7f2398101590 2024-06-18T13:03:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.579+0000 7f238ffff640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f238406e040 0x7f2384070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.579+0000 7f238dd9b640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f238406e040 0x7f2384070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.579+0000 7f238ffff640 1 -- 172.21.15.167:0/3560696935 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7f238808e9a0 con 0x7f2398101590 2024-06-18T13:03:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.579+0000 7f238ffff640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249] conn(0x7f2384077e30 0x7f238407a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.579+0000 7f238ffff640 1 -- 172.21.15.167:0/3560696935 --> [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f238801ee50 con 0x7f2384077e30 2024-06-18T13:03:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.579+0000 7f238ffff640 1 -- 172.21.15.167:0/3560696935 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f23880932c0 con 0x7f2398101590 2024-06-18T13:03:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.579+0000 7f238dd9b640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f238406e040 0x7f2384070500 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2380031440 tx=0x7f2380005b50 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.579+0000 7f2396d9d640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249] conn(0x7f2384077e30 0x7f238407a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.580+0000 7f2396d9d640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249] conn(0x7f2384077e30 0x7f238407a270 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:17.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.581+0000 7f238ffff640 1 -- 172.21.15.167:0/3560696935 <== osd.7 v2:172.21.15.179:6824/3253527249 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f238801ee50 con 0x7f2384077e30 2024-06-18T13:03:17.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.595+0000 7f239cffc640 1 -- 172.21.15.167:0/3560696935 --> [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2398102a70 con 0x7f2384077e30 2024-06-18T13:03:17.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.596+0000 7f238ffff640 1 -- 172.21.15.167:0/3560696935 <== osd.7 v2:172.21.15.179:6824/3253527249 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2398102a70 con 0x7f2384077e30 2024-06-18T13:03:17.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.596+0000 7f239cffc640 1 -- 172.21.15.167:0/3560696935 >> [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249] conn(0x7f2384077e30 msgr2=0x7f238407a270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:17.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.596+0000 7f239cffc640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.179:6824/3253527249,v1:172.21.15.179:6825/3253527249] conn(0x7f2384077e30 0x7f238407a270 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:17.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.596+0000 7f239cffc640 1 -- 172.21.15.167:0/3560696935 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f238406e040 msgr2=0x7f2384070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:17.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.596+0000 7f239cffc640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f238406e040 0x7f2384070500 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2380031440 tx=0x7f2380005b50 comp rx=0 tx=0).stop 2024-06-18T13:03:17.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.596+0000 7f239cffc640 1 -- 172.21.15.167:0/3560696935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2398101590 msgr2=0x7f23981971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:17.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.596+0000 7f239cffc640 1 --2- 172.21.15.167:0/3560696935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2398101590 0x7f23981971f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f238800ea60 tx=0x7f238800ef20 comp rx=0 tx=0).stop 2024-06-18T13:03:17.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.596+0000 7f2396d9d640 1 -- 172.21.15.167:0/3560696935 reap_dead start 2024-06-18T13:03:17.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.596+0000 7f239cffc640 1 -- 172.21.15.167:0/3560696935 shutdown_connections 2024-06-18T13:03:17.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.596+0000 7f239cffc640 1 -- 172.21.15.167:0/3560696935 >> 172.21.15.167:0/3560696935 conn(0x7f23980fd070 msgr2=0x7f23980fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:17.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.596+0000 7f239cffc640 1 -- 172.21.15.167:0/3560696935 shutdown_connections 2024-06-18T13:03:17.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:17.596+0000 7f239cffc640 1 -- 172.21.15.167:0/3560696935 wait complete. 2024-06-18T13:03:17.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:17 smithi179 ceph-mon[33312]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-18T13:03:18.356 INFO:teuthology.orchestra.run.smithi167.stdout:240518168591 2024-06-18T13:03:18.356 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-18T13:03:18.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.775+0000 7f5559770640 1 -- 172.21.15.167:0/3416045217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5554100490 msgr2=0x7f5554100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:18.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.775+0000 7f5559770640 1 --2- 172.21.15.167:0/3416045217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5554100490 0x7f5554100870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f553c009860 tx=0x7f553c02a340 comp rx=0 tx=0).stop 2024-06-18T13:03:18.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.775+0000 7f5559770640 1 -- 172.21.15.167:0/3416045217 shutdown_connections 2024-06-18T13:03:18.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.775+0000 7f5559770640 1 --2- 172.21.15.167:0/3416045217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55541055c0 0x7f555410d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:18.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.775+0000 7f5559770640 1 --2- 172.21.15.167:0/3416045217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5554100e40 0x7f5554104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:18.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.775+0000 7f5559770640 1 --2- 172.21.15.167:0/3416045217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5554100490 0x7f5554100870 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:18.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.775+0000 7f5559770640 1 -- 172.21.15.167:0/3416045217 >> 172.21.15.167:0/3416045217 conn(0x7f55540fbf50 msgr2=0x7f55540fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:18.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.775+0000 7f5559770640 1 -- 172.21.15.167:0/3416045217 shutdown_connections 2024-06-18T13:03:18.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.775+0000 7f5559770640 1 -- 172.21.15.167:0/3416045217 wait complete. 2024-06-18T13:03:18.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.775+0000 7f5559770640 1 Processor -- start 2024-06-18T13:03:18.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f5559770640 1 -- start start 2024-06-18T13:03:18.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f5559770640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5554100490 0x7f55541960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:18.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f5559770640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5554100e40 0x7f5554196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:18.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f5559770640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55541055c0 0x7f555419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:18.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f5559770640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f555410fac0 con 0x7f5554100e40 2024-06-18T13:03:18.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f5559770640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f555410f940 con 0x7f5554100490 2024-06-18T13:03:18.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f5559770640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f555410fc40 con 0x7f55541055c0 2024-06-18T13:03:18.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f55527fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5554100e40 0x7f5554196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:18.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f55537fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55541055c0 0x7f555419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:18.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f55537fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55541055c0 0x7f555419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:36536/0 (socket says 172.21.15.167:36536) 2024-06-18T13:03:18.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f55537fe640 1 -- 172.21.15.167:0/1721564443 learned_addr learned my addr 172.21.15.167:0/1721564443 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:18.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f5552ffd640 1 --2- 172.21.15.167:0/1721564443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5554100490 0x7f55541960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:18.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f55527fc640 1 -- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55541055c0 msgr2=0x7f555419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:18.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f55527fc640 1 --2- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55541055c0 0x7f555419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:18.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f55527fc640 1 -- 172.21.15.167:0/1721564443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5554100490 msgr2=0x7f55541960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:18.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f55527fc640 1 --2- 172.21.15.167:0/1721564443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5554100490 0x7f55541960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:18.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f55527fc640 1 -- 172.21.15.167:0/1721564443 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555419b080 con 0x7f5554100e40 2024-06-18T13:03:18.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.776+0000 7f55537fe640 1 --2- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55541055c0 0x7f555419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:03:18.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.777+0000 7f55527fc640 1 --2- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5554100e40 0x7f5554196610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5544005fd0 tx=0x7f554400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:18.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.777+0000 7f554bfff640 1 -- 172.21.15.167:0/1721564443 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554400bc00 con 0x7f5554100e40 2024-06-18T13:03:18.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.777+0000 7f5559770640 1 -- 172.21.15.167:0/1721564443 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555419b370 con 0x7f5554100e40 2024-06-18T13:03:18.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.777+0000 7f554bfff640 1 -- 172.21.15.167:0/1721564443 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f55440044a0 con 0x7f5554100e40 2024-06-18T13:03:18.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.777+0000 7f554bfff640 1 -- 172.21.15.167:0/1721564443 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5544003d90 con 0x7f5554100e40 2024-06-18T13:03:18.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.777+0000 7f5559770640 1 -- 172.21.15.167:0/1721564443 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5554069f40 con 0x7f5554100e40 2024-06-18T13:03:18.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.778+0000 7f5559770640 1 -- 172.21.15.167:0/1721564443 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55540fcf60 con 0x7f5554100e40 2024-06-18T13:03:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.778+0000 7f554bfff640 1 -- 172.21.15.167:0/1721564443 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f5544003f50 con 0x7f5554100e40 2024-06-18T13:03:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.779+0000 7f554bfff640 1 --2- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f554006e040 0x7f5540070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.779+0000 7f554bfff640 1 -- 172.21.15.167:0/1721564443 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7f554408fd40 con 0x7f5554100e40 2024-06-18T13:03:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.779+0000 7f5552ffd640 1 --2- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f554006e040 0x7f5540070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.779+0000 7f554bfff640 1 --2- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972] conn(0x7f5540077e30 0x7f554007a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.779+0000 7f554bfff640 1 -- 172.21.15.167:0/1721564443 --> [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5544019e50 con 0x7f5540077e30 2024-06-18T13:03:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.779+0000 7f554bfff640 1 -- 172.21.15.167:0/1721564443 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f5544092440 con 0x7f5554100e40 2024-06-18T13:03:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.779+0000 7f55537fe640 1 --2- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972] conn(0x7f5540077e30 0x7f554007a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.779+0000 7f5552ffd640 1 --2- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f554006e040 0x7f5540070500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f553c009830 tx=0x7f553c0058d0 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.779+0000 7f55537fe640 1 --2- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972] conn(0x7f5540077e30 0x7f554007a270 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:18.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.780+0000 7f554bfff640 1 -- 172.21.15.167:0/1721564443 <== osd.1 v2:172.21.15.167:6810/2010202972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5544019e50 con 0x7f5540077e30 2024-06-18T13:03:18.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.794+0000 7f5559770640 1 -- 172.21.15.167:0/1721564443 --> [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5554101970 con 0x7f5540077e30 2024-06-18T13:03:18.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.794+0000 7f554bfff640 1 -- 172.21.15.167:0/1721564443 <== osd.1 v2:172.21.15.167:6810/2010202972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5554101970 con 0x7f5540077e30 2024-06-18T13:03:18.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.795+0000 7f5559770640 1 -- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972] conn(0x7f5540077e30 msgr2=0x7f554007a270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:18.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.795+0000 7f5559770640 1 --2- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:6810/2010202972,v1:172.21.15.167:6811/2010202972] conn(0x7f5540077e30 0x7f554007a270 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:18.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.795+0000 7f5559770640 1 -- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f554006e040 msgr2=0x7f5540070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:18.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.795+0000 7f5559770640 1 --2- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f554006e040 0x7f5540070500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f553c009830 tx=0x7f553c0058d0 comp rx=0 tx=0).stop 2024-06-18T13:03:18.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.795+0000 7f5559770640 1 -- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5554100e40 msgr2=0x7f5554196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:18.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.795+0000 7f5559770640 1 --2- 172.21.15.167:0/1721564443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5554100e40 0x7f5554196610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5544005fd0 tx=0x7f554400ef60 comp rx=0 tx=0).stop 2024-06-18T13:03:18.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.795+0000 7f55537fe640 1 -- 172.21.15.167:0/1721564443 reap_dead start 2024-06-18T13:03:18.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.795+0000 7f5559770640 1 -- 172.21.15.167:0/1721564443 shutdown_connections 2024-06-18T13:03:18.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.795+0000 7f5559770640 1 -- 172.21.15.167:0/1721564443 >> 172.21.15.167:0/1721564443 conn(0x7f55540fbf50 msgr2=0x7f55540fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:18.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.795+0000 7f5559770640 1 -- 172.21.15.167:0/1721564443 shutdown_connections 2024-06-18T13:03:18.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.795+0000 7f5559770640 1 -- 172.21.15.167:0/1721564443 wait complete. 2024-06-18T13:03:18.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:03:18 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:03:18.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.975+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/2327685839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d841066d0 msgr2=0x7f8d8410e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:18.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.975+0000 7f8d8c7a2640 1 --2- 172.21.15.167:0/2327685839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d841066d0 0x7f8d8410e250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009ca0 tx=0x7f8d7402a8c0 comp rx=0 tx=0).stop 2024-06-18T13:03:18.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.975+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/2327685839 shutdown_connections 2024-06-18T13:03:18.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.975+0000 7f8d8c7a2640 1 --2- 172.21.15.167:0/2327685839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d841066d0 0x7f8d8410e250 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:18.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.975+0000 7f8d8c7a2640 1 --2- 172.21.15.167:0/2327685839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d84101f50 0x7f8d84105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:18.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.975+0000 7f8d8c7a2640 1 --2- 172.21.15.167:0/2327685839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d841015a0 0x7f8d84101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:18.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.975+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/2327685839 >> 172.21.15.167:0/2327685839 conn(0x7f8d840fd060 msgr2=0x7f8d840ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:18.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.975+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/2327685839 shutdown_connections 2024-06-18T13:03:18.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.975+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/2327685839 wait complete. 2024-06-18T13:03:18.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8c7a2640 1 Processor -- start 2024-06-18T13:03:18.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8c7a2640 1 -- start start 2024-06-18T13:03:18.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8c7a2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d841015a0 0x7f8d84073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:18.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8c7a2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d84101f50 0x7f8d840741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:18.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8c7a2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d841066d0 0x7f8d84074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:18.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8c7a2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8d84110b60 con 0x7f8d841066d0 2024-06-18T13:03:18.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8c7a2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841109e0 con 0x7f8d84101f50 2024-06-18T13:03:18.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8c7a2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8d84110ce0 con 0x7f8d841015a0 2024-06-18T13:03:18.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8a53e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d841015a0 0x7f8d84073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:18.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8a53e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d841015a0 0x7f8d84073ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:36552/0 (socket says 172.21.15.167:36552) 2024-06-18T13:03:18.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8ad3f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d841066d0 0x7f8d84074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:18.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8a53e640 1 -- 172.21.15.167:0/3325842246 learned_addr learned my addr 172.21.15.167:0/3325842246 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:18.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d89d3d640 1 --2- 172.21.15.167:0/3325842246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d84101f50 0x7f8d840741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:18.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8a53e640 1 -- 172.21.15.167:0/3325842246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d84101f50 msgr2=0x7f8d840741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:18.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8a53e640 1 --2- 172.21.15.167:0/3325842246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d84101f50 0x7f8d840741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:18.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8a53e640 1 -- 172.21.15.167:0/3325842246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d841066d0 msgr2=0x7f8d84074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:18.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8a53e640 1 --2- 172.21.15.167:0/3325842246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d841066d0 0x7f8d84074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:18.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8a53e640 1 -- 172.21.15.167:0/3325842246 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d841a55b0 con 0x7f8d841015a0 2024-06-18T13:03:18.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.976+0000 7f8d8a53e640 1 --2- 172.21.15.167:0/3325842246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d841015a0 0x7f8d84073ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00d950 tx=0x7f8d7c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:18.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.977+0000 7f8d837fe640 1 -- 172.21.15.167:0/3325842246 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c017070 con 0x7f8d841015a0 2024-06-18T13:03:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.977+0000 7f8d837fe640 1 -- 172.21.15.167:0/3325842246 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f8d7c009c30 con 0x7f8d841015a0 2024-06-18T13:03:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.977+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/3325842246 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d841a58a0 con 0x7f8d841015a0 2024-06-18T13:03:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.977+0000 7f8d837fe640 1 -- 172.21.15.167:0/3325842246 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c00f620 con 0x7f8d841015a0 2024-06-18T13:03:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.977+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/3325842246 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d841a5de0 con 0x7f8d841015a0 2024-06-18T13:03:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.977+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/3325842246 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8d840fdfc0 con 0x7f8d841015a0 2024-06-18T13:03:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.978+0000 7f8d837fe640 1 -- 172.21.15.167:0/3325842246 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f8d7c010460 con 0x7f8d841015a0 2024-06-18T13:03:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.978+0000 7f8d837fe640 1 --2- 172.21.15.167:0/3325842246 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f8d6c06e040 0x7f8d6c070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.978+0000 7f8d837fe640 1 -- 172.21.15.167:0/3325842246 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7f8d7c08e210 con 0x7f8d841015a0 2024-06-18T13:03:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.978+0000 7f8d89d3d640 1 --2- 172.21.15.167:0/3325842246 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f8d6c06e040 0x7f8d6c070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.979+0000 7f8d837fe640 1 --2- 172.21.15.167:0/3325842246 >> [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992] conn(0x7f8d6c077e30 0x7f8d6c07a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.979+0000 7f8d837fe640 1 -- 172.21.15.167:0/3325842246 --> [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d7c004470 con 0x7f8d6c077e30 2024-06-18T13:03:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.979+0000 7f8d837fe640 1 -- 172.21.15.167:0/3325842246 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f8d7c095050 con 0x7f8d841015a0 2024-06-18T13:03:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.979+0000 7f8d89d3d640 1 --2- 172.21.15.167:0/3325842246 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f8d6c06e040 0x7f8d6c070500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d84075730 tx=0x7f8d78005f90 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:18.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.979+0000 7f8d8ad3f640 1 --2- 172.21.15.167:0/3325842246 >> [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992] conn(0x7f8d6c077e30 0x7f8d6c07a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:18.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.979+0000 7f8d8ad3f640 1 --2- 172.21.15.167:0/3325842246 >> [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992] conn(0x7f8d6c077e30 0x7f8d6c07a270 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:18.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.980+0000 7f8d837fe640 1 -- 172.21.15.167:0/3325842246 <== osd.5 v2:172.21.15.179:6808/1550696992 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d7c004470 con 0x7f8d6c077e30 2024-06-18T13:03:18.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.994+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/3325842246 --> [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8d84102a80 con 0x7f8d6c077e30 2024-06-18T13:03:18.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.995+0000 7f8d837fe640 1 -- 172.21.15.167:0/3325842246 <== osd.5 v2:172.21.15.179:6808/1550696992 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8d84102a80 con 0x7f8d6c077e30 2024-06-18T13:03:18.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.995+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/3325842246 >> [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992] conn(0x7f8d6c077e30 msgr2=0x7f8d6c07a270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:18.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.995+0000 7f8d8c7a2640 1 --2- 172.21.15.167:0/3325842246 >> [v2:172.21.15.179:6808/1550696992,v1:172.21.15.179:6809/1550696992] conn(0x7f8d6c077e30 0x7f8d6c07a270 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:18.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.995+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/3325842246 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f8d6c06e040 msgr2=0x7f8d6c070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:18.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.995+0000 7f8d8c7a2640 1 --2- 172.21.15.167:0/3325842246 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f8d6c06e040 0x7f8d6c070500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d84075730 tx=0x7f8d78005f90 comp rx=0 tx=0).stop 2024-06-18T13:03:18.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.995+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/3325842246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d841015a0 msgr2=0x7f8d84073ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:18.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.995+0000 7f8d8c7a2640 1 --2- 172.21.15.167:0/3325842246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d841015a0 0x7f8d84073ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00d950 tx=0x7f8d7c00de10 comp rx=0 tx=0).stop 2024-06-18T13:03:18.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.995+0000 7f8d8ad3f640 1 -- 172.21.15.167:0/3325842246 reap_dead start 2024-06-18T13:03:18.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.995+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/3325842246 shutdown_connections 2024-06-18T13:03:18.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.995+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/3325842246 >> 172.21.15.167:0/3325842246 conn(0x7f8d840fd060 msgr2=0x7f8d840fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:18.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.995+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/3325842246 shutdown_connections 2024-06-18T13:03:18.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:18.995+0000 7f8d8c7a2640 1 -- 172.21.15.167:0/3325842246 wait complete. 2024-06-18T13:03:19.452 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:19 smithi167 ceph-mon[37701]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-06-18T13:03:19.452 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:19 smithi167 ceph-mon[29496]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-06-18T13:03:19.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:19 smithi179 ceph-mon[33312]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-06-18T13:03:19.715 INFO:teuthology.orchestra.run.smithi167.stdout:167503724569 2024-06-18T13:03:19.715 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-18T13:03:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:20 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:03:20.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:20 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:03:20.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:20 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:03:20.674 INFO:teuthology.orchestra.run.smithi167.stdout:55834574891 2024-06-18T13:03:20.675 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-18T13:03:20.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.843+0000 7f2287a29640 1 -- 172.21.15.167:0/4057645289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2280101e50 msgr2=0x7f2280102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:20.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.843+0000 7f2287a29640 1 --2- 172.21.15.167:0/4057645289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2280101e50 0x7f2280102290 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f227c009930 tx=0x7f227c02a420 comp rx=0 tx=0).stop 2024-06-18T13:03:20.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.843+0000 7f2287a29640 1 -- 172.21.15.167:0/4057645289 shutdown_connections 2024-06-18T13:03:20.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.843+0000 7f2287a29640 1 --2- 172.21.15.167:0/4057645289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22801027d0 0x7f228010d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:20.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.843+0000 7f2287a29640 1 --2- 172.21.15.167:0/4057645289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2280101e50 0x7f2280102290 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:20.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.843+0000 7f2287a29640 1 --2- 172.21.15.167:0/4057645289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2280100560 0x7f2280100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:20.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.843+0000 7f2287a29640 1 -- 172.21.15.167:0/4057645289 >> 172.21.15.167:0/4057645289 conn(0x7f22800fbf80 msgr2=0x7f22800fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:20.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.843+0000 7f2287a29640 1 -- 172.21.15.167:0/4057645289 shutdown_connections 2024-06-18T13:03:20.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.844+0000 7f2287a29640 1 -- 172.21.15.167:0/4057645289 wait complete. 2024-06-18T13:03:20.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.844+0000 7f2287a29640 1 Processor -- start 2024-06-18T13:03:20.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.844+0000 7f2287a29640 1 -- start start 2024-06-18T13:03:20.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.844+0000 7f2287a29640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2280100560 0x7f22801980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:20.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.844+0000 7f2287a29640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2280101e50 0x7f22801985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:20.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.844+0000 7f2287a29640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22801027d0 0x7f22801a3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:20.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.844+0000 7f2287a29640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f228010fbc0 con 0x7f22801027d0 2024-06-18T13:03:20.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f2287a29640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f228010fa40 con 0x7f2280100560 2024-06-18T13:03:20.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f2287a29640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f228010fd40 con 0x7f2280101e50 2024-06-18T13:03:20.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f2285fc6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22801027d0 0x7f22801a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:20.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f2284fc4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2280101e50 0x7f22801985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:20.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f2285fc6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22801027d0 0x7f22801a3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:50138/0 (socket says 172.21.15.167:50138) 2024-06-18T13:03:20.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f2285fc6640 1 -- 172.21.15.167:0/1939523785 learned_addr learned my addr 172.21.15.167:0/1939523785 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:20.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f22857c5640 1 --2- 172.21.15.167:0/1939523785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2280100560 0x7f22801980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:20.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f2285fc6640 1 -- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2280101e50 msgr2=0x7f22801985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:20.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f2285fc6640 1 --2- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2280101e50 0x7f22801985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:20.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f2285fc6640 1 -- 172.21.15.167:0/1939523785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2280100560 msgr2=0x7f22801980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:20.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f2285fc6640 1 --2- 172.21.15.167:0/1939523785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2280100560 0x7f22801980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:20.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f2285fc6640 1 -- 172.21.15.167:0/1939523785 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22801a3660 con 0x7f22801027d0 2024-06-18T13:03:20.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f2285fc6640 1 --2- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22801027d0 0x7f22801a3120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f227800efd0 tx=0x7f227800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:20.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f22727fc640 1 -- 172.21.15.167:0/1939523785 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278003b70 con 0x7f22801027d0 2024-06-18T13:03:20.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f2287a29640 1 -- 172.21.15.167:0/1939523785 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22801a3830 con 0x7f22801027d0 2024-06-18T13:03:20.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f22727fc640 1 -- 172.21.15.167:0/1939523785 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f2278003d10 con 0x7f22801027d0 2024-06-18T13:03:20.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f22727fc640 1 -- 172.21.15.167:0/1939523785 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278010770 con 0x7f22801027d0 2024-06-18T13:03:20.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.845+0000 7f2287a29640 1 -- 172.21.15.167:0/1939523785 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22801a3cf0 con 0x7f22801027d0 2024-06-18T13:03:20.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.847+0000 7f226bfff640 1 -- 172.21.15.167:0/1939523785 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2280062070 con 0x7f22801027d0 2024-06-18T13:03:20.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.847+0000 7f22727fc640 1 -- 172.21.15.167:0/1939523785 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f2278010910 con 0x7f22801027d0 2024-06-18T13:03:20.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.848+0000 7f22727fc640 1 --2- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f226c06e040 0x7f226c070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:20.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.848+0000 7f22727fc640 1 -- 172.21.15.167:0/1939523785 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7f227808ea50 con 0x7f22801027d0 2024-06-18T13:03:20.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.848+0000 7f22857c5640 1 --2- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f226c06e040 0x7f226c070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:20.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.848+0000 7f22727fc640 1 --2- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921] conn(0x7f226c077e30 0x7f226c07a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:20.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.848+0000 7f22727fc640 1 -- 172.21.15.167:0/1939523785 --> [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f226c07a940 con 0x7f226c077e30 2024-06-18T13:03:20.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.848+0000 7f22727fc640 1 -- 172.21.15.167:0/1939523785 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f22780932c0 con 0x7f22801027d0 2024-06-18T13:03:20.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.848+0000 7f2284fc4640 1 --2- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921] conn(0x7f226c077e30 0x7f226c07a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:20.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.848+0000 7f22857c5640 1 --2- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f226c06e040 0x7f226c070500 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2274009990 tx=0x7f2274009110 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:20.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.848+0000 7f2284fc4640 1 --2- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921] conn(0x7f226c077e30 0x7f226c07a270 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:20.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.849+0000 7f22727fc640 1 -- 172.21.15.167:0/1939523785 <== osd.2 v2:172.21.15.167:6818/183062921 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f226c07a940 con 0x7f226c077e30 2024-06-18T13:03:20.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.865+0000 7f226bfff640 1 -- 172.21.15.167:0/1939523785 --> [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2280062470 con 0x7f226c077e30 2024-06-18T13:03:20.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.866+0000 7f22727fc640 1 -- 172.21.15.167:0/1939523785 <== osd.2 v2:172.21.15.167:6818/183062921 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2280062470 con 0x7f226c077e30 2024-06-18T13:03:20.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.866+0000 7f226bfff640 1 -- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921] conn(0x7f226c077e30 msgr2=0x7f226c07a270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:20.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.866+0000 7f226bfff640 1 --2- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:6818/183062921,v1:172.21.15.167:6819/183062921] conn(0x7f226c077e30 0x7f226c07a270 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:20.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.866+0000 7f226bfff640 1 -- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f226c06e040 msgr2=0x7f226c070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:20.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.866+0000 7f226bfff640 1 --2- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f226c06e040 0x7f226c070500 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2274009990 tx=0x7f2274009110 comp rx=0 tx=0).stop 2024-06-18T13:03:20.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.866+0000 7f226bfff640 1 -- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22801027d0 msgr2=0x7f22801a3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:20.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.866+0000 7f226bfff640 1 --2- 172.21.15.167:0/1939523785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22801027d0 0x7f22801a3120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f227800efd0 tx=0x7f227800c6a0 comp rx=0 tx=0).stop 2024-06-18T13:03:20.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.866+0000 7f2285fc6640 1 -- 172.21.15.167:0/1939523785 reap_dead start 2024-06-18T13:03:20.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.866+0000 7f226bfff640 1 -- 172.21.15.167:0/1939523785 shutdown_connections 2024-06-18T13:03:20.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.866+0000 7f226bfff640 1 -- 172.21.15.167:0/1939523785 >> 172.21.15.167:0/1939523785 conn(0x7f22800fbf80 msgr2=0x7f22800fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:20.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.866+0000 7f226bfff640 1 -- 172.21.15.167:0/1939523785 shutdown_connections 2024-06-18T13:03:20.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:20.866+0000 7f226bfff640 1 -- 172.21.15.167:0/1939523785 wait complete. 2024-06-18T13:03:21.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:21 smithi167 ceph-mon[37701]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-18T13:03:21.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:21 smithi167 ceph-mon[29496]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-18T13:03:21.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:21 smithi179 ceph-mon[33312]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-18T13:03:21.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.762+0000 7f13d209a640 1 -- 172.21.15.167:0/654553392 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13cc101590 msgr2=0x7f13cc101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:21.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.762+0000 7f13d209a640 1 --2- 172.21.15.167:0/654553392 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13cc101590 0x7f13cc101970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009ca0 tx=0x7f13bc02a8c0 comp rx=0 tx=0).stop 2024-06-18T13:03:21.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.762+0000 7f13d209a640 1 -- 172.21.15.167:0/654553392 shutdown_connections 2024-06-18T13:03:21.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.762+0000 7f13d209a640 1 --2- 172.21.15.167:0/654553392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13cc1066c0 0x7f13cc10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:21.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.762+0000 7f13d209a640 1 --2- 172.21.15.167:0/654553392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13cc101f40 0x7f13cc105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:21.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.762+0000 7f13d209a640 1 --2- 172.21.15.167:0/654553392 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13cc101590 0x7f13cc101970 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:21.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.762+0000 7f13d209a640 1 -- 172.21.15.167:0/654553392 >> 172.21.15.167:0/654553392 conn(0x7f13cc0fd070 msgr2=0x7f13cc0ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:21.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.762+0000 7f13d209a640 1 -- 172.21.15.167:0/654553392 shutdown_connections 2024-06-18T13:03:21.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.762+0000 7f13d209a640 1 -- 172.21.15.167:0/654553392 wait complete. 2024-06-18T13:03:21.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.763+0000 7f13d209a640 1 Processor -- start 2024-06-18T13:03:21.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.763+0000 7f13d209a640 1 -- start start 2024-06-18T13:03:21.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.763+0000 7f13d209a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13cc101590 0x7f13cc073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:21.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.763+0000 7f13d209a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13cc101f40 0x7f13cc0741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:21.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.763+0000 7f13d209a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13cc1066c0 0x7f13cc074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:21.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.763+0000 7f13d209a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc110bb0 con 0x7f13cc101f40 2024-06-18T13:03:21.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.763+0000 7f13d209a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc110a30 con 0x7f13cc101590 2024-06-18T13:03:21.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.763+0000 7f13d209a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f13cc110d30 con 0x7f13cc1066c0 2024-06-18T13:03:21.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.763+0000 7f13caffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13cc101f40 0x7f13cc0741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:21.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.763+0000 7f13cbfff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13cc1066c0 0x7f13cc074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:21.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.763+0000 7f13caffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13cc101f40 0x7f13cc0741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:50164/0 (socket says 172.21.15.167:50164) 2024-06-18T13:03:21.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.763+0000 7f13caffd640 1 -- 172.21.15.167:0/2362143255 learned_addr learned my addr 172.21.15.167:0/2362143255 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:21.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.764+0000 7f13cb7fe640 1 --2- 172.21.15.167:0/2362143255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13cc101590 0x7f13cc073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:21.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.764+0000 7f13caffd640 1 -- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13cc1066c0 msgr2=0x7f13cc074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:21.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.764+0000 7f13caffd640 1 --2- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13cc1066c0 0x7f13cc074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:21.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.764+0000 7f13caffd640 1 -- 172.21.15.167:0/2362143255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13cc101590 msgr2=0x7f13cc073ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:21.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.764+0000 7f13caffd640 1 --2- 172.21.15.167:0/2362143255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13cc101590 0x7f13cc073ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:21.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.764+0000 7f13caffd640 1 -- 172.21.15.167:0/2362143255 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc1a5730 con 0x7f13cc101f40 2024-06-18T13:03:21.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.764+0000 7f13cbfff640 1 --2- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13cc1066c0 0x7f13cc074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:03:21.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.764+0000 7f13caffd640 1 --2- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13cc101f40 0x7f13cc0741e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f13c000ea60 tx=0x7f13c000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:21.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.764+0000 7f13c8ff9640 1 -- 172.21.15.167:0/2362143255 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c000cde0 con 0x7f13cc101f40 2024-06-18T13:03:21.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.764+0000 7f13c8ff9640 1 -- 172.21.15.167:0/2362143255 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f13c00035f0 con 0x7f13cc101f40 2024-06-18T13:03:21.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.764+0000 7f13d209a640 1 -- 172.21.15.167:0/2362143255 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13cc1a5a20 con 0x7f13cc101f40 2024-06-18T13:03:21.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.764+0000 7f13c8ff9640 1 -- 172.21.15.167:0/2362143255 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c00044a0 con 0x7f13cc101f40 2024-06-18T13:03:21.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.764+0000 7f13d209a640 1 -- 172.21.15.167:0/2362143255 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13cc1a5f60 con 0x7f13cc101f40 2024-06-18T13:03:21.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.765+0000 7f13d209a640 1 -- 172.21.15.167:0/2362143255 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f13cc0fe0f0 con 0x7f13cc101f40 2024-06-18T13:03:21.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.766+0000 7f13c8ff9640 1 -- 172.21.15.167:0/2362143255 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f13c0004000 con 0x7f13cc101f40 2024-06-18T13:03:21.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.766+0000 7f13c8ff9640 1 --2- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f13b406e040 0x7f13b4070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:21.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.766+0000 7f13c8ff9640 1 -- 172.21.15.167:0/2362143255 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7f13c008e8e0 con 0x7f13cc101f40 2024-06-18T13:03:21.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.766+0000 7f13cb7fe640 1 --2- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f13b406e040 0x7f13b4070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:21.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.766+0000 7f13c8ff9640 1 --2- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419] conn(0x7f13b4077e30 0x7f13b407a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:21.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.766+0000 7f13c8ff9640 1 -- 172.21.15.167:0/2362143255 --> [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13c00042f0 con 0x7f13b4077e30 2024-06-18T13:03:21.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.766+0000 7f13c8ff9640 1 -- 172.21.15.167:0/2362143255 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f13c00932c0 con 0x7f13cc101f40 2024-06-18T13:03:21.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.766+0000 7f13cbfff640 1 --2- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419] conn(0x7f13b4077e30 0x7f13b407a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:21.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.767+0000 7f13cb7fe640 1 --2- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f13b406e040 0x7f13b4070500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009430 tx=0x7f13bc009360 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:21.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.767+0000 7f13cbfff640 1 --2- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419] conn(0x7f13b4077e30 0x7f13b407a270 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:21.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.767+0000 7f13c8ff9640 1 -- 172.21.15.167:0/2362143255 <== osd.0 v2:172.21.15.167:6802/3001811419 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13c00042f0 con 0x7f13b4077e30 2024-06-18T13:03:21.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.782+0000 7f13d209a640 1 -- 172.21.15.167:0/2362143255 --> [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f13cc102a70 con 0x7f13b4077e30 2024-06-18T13:03:21.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.782+0000 7f13c8ff9640 1 -- 172.21.15.167:0/2362143255 <== osd.0 v2:172.21.15.167:6802/3001811419 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f13cc102a70 con 0x7f13b4077e30 2024-06-18T13:03:21.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.782+0000 7f13d209a640 1 -- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419] conn(0x7f13b4077e30 msgr2=0x7f13b407a270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:21.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.782+0000 7f13d209a640 1 --2- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:6802/3001811419,v1:172.21.15.167:6803/3001811419] conn(0x7f13b4077e30 0x7f13b407a270 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:21.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.783+0000 7f13d209a640 1 -- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f13b406e040 msgr2=0x7f13b4070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:21.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.783+0000 7f13d209a640 1 --2- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f13b406e040 0x7f13b4070500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009430 tx=0x7f13bc009360 comp rx=0 tx=0).stop 2024-06-18T13:03:21.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.783+0000 7f13d209a640 1 -- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13cc101f40 msgr2=0x7f13cc0741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:21.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.783+0000 7f13d209a640 1 --2- 172.21.15.167:0/2362143255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13cc101f40 0x7f13cc0741e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f13c000ea60 tx=0x7f13c000ef20 comp rx=0 tx=0).stop 2024-06-18T13:03:21.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.783+0000 7f13cbfff640 1 -- 172.21.15.167:0/2362143255 reap_dead start 2024-06-18T13:03:21.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.783+0000 7f13d209a640 1 -- 172.21.15.167:0/2362143255 shutdown_connections 2024-06-18T13:03:21.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.783+0000 7f13d209a640 1 -- 172.21.15.167:0/2362143255 >> 172.21.15.167:0/2362143255 conn(0x7f13cc0fd070 msgr2=0x7f13cc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:21.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.783+0000 7f13d209a640 1 -- 172.21.15.167:0/2362143255 shutdown_connections 2024-06-18T13:03:21.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:21.783+0000 7f13d209a640 1 -- 172.21.15.167:0/2362143255 wait complete. 2024-06-18T13:03:22.206 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:22.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.391+0000 7feaa2467640 1 -- 172.21.15.167:0/1015823342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea9c105350 msgr2=0x7fea9c105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:22.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.391+0000 7feaa2467640 1 --2- 172.21.15.167:0/1015823342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea9c105350 0x7fea9c105730 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009900 tx=0x7fea8c02a480 comp rx=0 tx=0).stop 2024-06-18T13:03:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.391+0000 7feaa2467640 1 -- 172.21.15.167:0/1015823342 shutdown_connections 2024-06-18T13:03:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.391+0000 7feaa2467640 1 --2- 172.21.15.167:0/1015823342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea9c1018c0 0x7fea9c10d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.391+0000 7feaa2467640 1 --2- 172.21.15.167:0/1015823342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea9c100f40 0x7fea9c101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.391+0000 7feaa2467640 1 --2- 172.21.15.167:0/1015823342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea9c105350 0x7fea9c105730 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.391+0000 7feaa2467640 1 -- 172.21.15.167:0/1015823342 >> 172.21.15.167:0/1015823342 conn(0x7fea9c0769c0 msgr2=0x7fea9c076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.391+0000 7feaa2467640 1 -- 172.21.15.167:0/1015823342 shutdown_connections 2024-06-18T13:03:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.391+0000 7feaa2467640 1 -- 172.21.15.167:0/1015823342 wait complete. 2024-06-18T13:03:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.391+0000 7feaa2467640 1 Processor -- start 2024-06-18T13:03:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.391+0000 7feaa2467640 1 -- start start 2024-06-18T13:03:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7feaa2467640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea9c100f40 0x7fea9c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7feaa2467640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea9c1018c0 0x7fea9c1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7feaa2467640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea9c105350 0x7fea9c19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:22.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7feaa2467640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c110c20 con 0x7fea9c105350 2024-06-18T13:03:22.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7feaa2467640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c110aa0 con 0x7fea9c100f40 2024-06-18T13:03:22.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7feaa2467640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fea9c110da0 con 0x7fea9c1018c0 2024-06-18T13:03:22.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7fea93fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea9c1018c0 0x7fea9c1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:22.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7fea93fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea9c1018c0 0x7fea9c1976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:36586/0 (socket says 172.21.15.167:36586) 2024-06-18T13:03:22.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7fea93fff640 1 -- 172.21.15.167:0/4192534445 learned_addr learned my addr 172.21.15.167:0/4192534445 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:22.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7feaa0a04640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea9c105350 0x7fea9c19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:22.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7fea9bfff640 1 --2- 172.21.15.167:0/4192534445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea9c100f40 0x7fea9c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:22.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7fea93fff640 1 -- 172.21.15.167:0/4192534445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea9c100f40 msgr2=0x7fea9c197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:22.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7fea93fff640 1 --2- 172.21.15.167:0/4192534445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea9c100f40 0x7fea9c197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:22.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7fea93fff640 1 -- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea9c105350 msgr2=0x7fea9c19ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:22.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7fea93fff640 1 --2- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea9c105350 0x7fea9c19ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:22.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7fea93fff640 1 -- 172.21.15.167:0/4192534445 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c19c110 con 0x7fea9c1018c0 2024-06-18T13:03:22.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7feaa0a04640 1 --2- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea9c105350 0x7fea9c19ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:03:22.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7fea93fff640 1 --2- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea9c1018c0 0x7fea9c1976a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fea8400c890 tx=0x7fea8400cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:22.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7fea99ffb640 1 -- 172.21.15.167:0/4192534445 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84004220 con 0x7fea9c1018c0 2024-06-18T13:03:22.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7fea99ffb640 1 -- 172.21.15.167:0/4192534445 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fea84007e30 con 0x7fea9c1018c0 2024-06-18T13:03:22.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.392+0000 7feaa2467640 1 -- 172.21.15.167:0/4192534445 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9c19c400 con 0x7fea9c1018c0 2024-06-18T13:03:22.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.393+0000 7fea99ffb640 1 -- 172.21.15.167:0/4192534445 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea840047a0 con 0x7fea9c1018c0 2024-06-18T13:03:22.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.393+0000 7feaa2467640 1 -- 172.21.15.167:0/4192534445 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9c1a3ce0 con 0x7fea9c1018c0 2024-06-18T13:03:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.393+0000 7feaa2467640 1 -- 172.21.15.167:0/4192534445 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea9c0620b0 con 0x7fea9c1018c0 2024-06-18T13:03:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.394+0000 7fea99ffb640 1 -- 172.21.15.167:0/4192534445 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7fea840079b0 con 0x7fea9c1018c0 2024-06-18T13:03:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.394+0000 7fea99ffb640 1 --2- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fea8806e040 0x7fea88070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.395+0000 7fea99ffb640 1 -- 172.21.15.167:0/4192534445 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7fea8408e5b0 con 0x7fea9c1018c0 2024-06-18T13:03:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.395+0000 7fea9bfff640 1 --2- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fea8806e040 0x7fea88070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.395+0000 7fea99ffb640 1 --2- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547] conn(0x7fea88077eb0 0x7fea8807a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:22.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.395+0000 7fea99ffb640 1 -- 172.21.15.167:0/4192534445 --> [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea84012ec0 con 0x7fea88077eb0 2024-06-18T13:03:22.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.395+0000 7fea99ffb640 1 -- 172.21.15.167:0/4192534445 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fea8408ea90 con 0x7fea9c1018c0 2024-06-18T13:03:22.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.395+0000 7feaa0a04640 1 --2- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547] conn(0x7fea88077eb0 0x7fea8807a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:22.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.395+0000 7fea9bfff640 1 --2- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fea8806e040 0x7fea88070500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fea8c031440 tx=0x7fea8c005b50 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:22.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.395+0000 7feaa0a04640 1 --2- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547] conn(0x7fea88077eb0 0x7fea8807a2f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:22.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.396+0000 7fea99ffb640 1 -- 172.21.15.167:0/4192534445 <== osd.3 v2:172.21.15.167:6826/2828000547 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea84012ec0 con 0x7fea88077eb0 2024-06-18T13:03:22.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.413+0000 7feaa2467640 1 -- 172.21.15.167:0/4192534445 --> [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea9c0624b0 con 0x7fea88077eb0 2024-06-18T13:03:22.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.413+0000 7fea99ffb640 1 -- 172.21.15.167:0/4192534445 <== osd.3 v2:172.21.15.167:6826/2828000547 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea9c0624b0 con 0x7fea88077eb0 2024-06-18T13:03:22.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.414+0000 7feaa2467640 1 -- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547] conn(0x7fea88077eb0 msgr2=0x7fea8807a2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:22.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.414+0000 7feaa2467640 1 --2- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:6826/2828000547,v1:172.21.15.167:6827/2828000547] conn(0x7fea88077eb0 0x7fea8807a2f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:22.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.414+0000 7feaa2467640 1 -- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fea8806e040 msgr2=0x7fea88070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:22.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.414+0000 7feaa2467640 1 --2- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fea8806e040 0x7fea88070500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fea8c031440 tx=0x7fea8c005b50 comp rx=0 tx=0).stop 2024-06-18T13:03:22.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.414+0000 7feaa2467640 1 -- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea9c1018c0 msgr2=0x7fea9c1976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:22.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.414+0000 7feaa2467640 1 --2- 172.21.15.167:0/4192534445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea9c1018c0 0x7fea9c1976a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fea8400c890 tx=0x7fea8400cd50 comp rx=0 tx=0).stop 2024-06-18T13:03:22.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.414+0000 7feaa0a04640 1 -- 172.21.15.167:0/4192534445 reap_dead start 2024-06-18T13:03:22.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.414+0000 7feaa2467640 1 -- 172.21.15.167:0/4192534445 shutdown_connections 2024-06-18T13:03:22.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.414+0000 7feaa2467640 1 -- 172.21.15.167:0/4192534445 >> 172.21.15.167:0/4192534445 conn(0x7fea9c0769c0 msgr2=0x7fea9c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:22.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.414+0000 7feaa2467640 1 -- 172.21.15.167:0/4192534445 shutdown_connections 2024-06-18T13:03:22.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:22.414+0000 7feaa2467640 1 -- 172.21.15.167:0/4192534445 wait complete. 2024-06-18T13:03:23.316 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:23 smithi167 ceph-mon[29496]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-18T13:03:23.316 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:23 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:03:23.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:23 smithi167 ceph-mon[37701]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-18T13:03:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:23 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:03:23.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:23 smithi179 ceph-mon[33312]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-18T13:03:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:23 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:03:23.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.786+0000 7fa8ca58b640 1 -- 172.21.15.167:0/2383545784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8c4100490 msgr2=0x7fa8c4100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:23.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.786+0000 7fa8ca58b640 1 --2- 172.21.15.167:0/2383545784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8c4100490 0x7fa8c4100870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009900 tx=0x7fa8b402a410 comp rx=0 tx=0).stop 2024-06-18T13:03:23.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.786+0000 7fa8ca58b640 1 -- 172.21.15.167:0/2383545784 shutdown_connections 2024-06-18T13:03:23.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.786+0000 7fa8ca58b640 1 --2- 172.21.15.167:0/2383545784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8c41055c0 0x7fa8c410d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:23.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.786+0000 7fa8ca58b640 1 --2- 172.21.15.167:0/2383545784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8c4100e40 0x7fa8c4104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:23.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.786+0000 7fa8ca58b640 1 --2- 172.21.15.167:0/2383545784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8c4100490 0x7fa8c4100870 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:23.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.786+0000 7fa8ca58b640 1 -- 172.21.15.167:0/2383545784 >> 172.21.15.167:0/2383545784 conn(0x7fa8c40fbf50 msgr2=0x7fa8c40fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:23.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.786+0000 7fa8ca58b640 1 -- 172.21.15.167:0/2383545784 shutdown_connections 2024-06-18T13:03:23.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.786+0000 7fa8ca58b640 1 -- 172.21.15.167:0/2383545784 wait complete. 2024-06-18T13:03:23.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.787+0000 7fa8ca58b640 1 Processor -- start 2024-06-18T13:03:23.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.787+0000 7fa8ca58b640 1 -- start start 2024-06-18T13:03:23.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.787+0000 7fa8ca58b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8c4100490 0x7fa8c4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:23.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.787+0000 7fa8ca58b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8c4100e40 0x7fa8c4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:23.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.787+0000 7fa8ca58b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8c41055c0 0x7fa8c41a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:23.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.787+0000 7fa8ca58b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c410fa40 con 0x7fa8c4100490 2024-06-18T13:03:23.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.787+0000 7fa8ca58b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c410f8c0 con 0x7fa8c4100e40 2024-06-18T13:03:23.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.787+0000 7fa8ca58b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c410fbc0 con 0x7fa8c41055c0 2024-06-18T13:03:23.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8c4100490 0x7fa8c4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:23.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8c4100490 0x7fa8c4197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53706/0 (socket says 172.21.15.167:53706) 2024-06-18T13:03:23.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c3fff640 1 -- 172.21.15.167:0/1614592551 learned_addr learned my addr 172.21.15.167:0/1614592551 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:23.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c8b28640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8c41055c0 0x7fa8c41a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:23.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c37fe640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8c4100e40 0x7fa8c4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:23.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c3fff640 1 -- 172.21.15.167:0/1614592551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8c41055c0 msgr2=0x7fa8c41a3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:23.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c3fff640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8c41055c0 0x7fa8c41a3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:23.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c3fff640 1 -- 172.21.15.167:0/1614592551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8c4100e40 msgr2=0x7fa8c4198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:23.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c3fff640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8c4100e40 0x7fa8c4198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:23.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c3fff640 1 -- 172.21.15.167:0/1614592551 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c41a35b0 con 0x7fa8c4100490 2024-06-18T13:03:23.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c8b28640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8c41055c0 0x7fa8c41a3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:03:23.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c3fff640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8c4100490 0x7fa8c4197ff0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4005940 tx=0x7fa8b4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:23.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c17fa640 1 -- 172.21.15.167:0/1614592551 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b403f070 con 0x7fa8c4100490 2024-06-18T13:03:23.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c17fa640 1 -- 172.21.15.167:0/1614592551 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fa8b4005590 con 0x7fa8c4100490 2024-06-18T13:03:23.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8ca58b640 1 -- 172.21.15.167:0/1614592551 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c41a3780 con 0x7fa8c4100490 2024-06-18T13:03:23.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8c17fa640 1 -- 172.21.15.167:0/1614592551 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4035030 con 0x7fa8c4100490 2024-06-18T13:03:23.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.788+0000 7fa8ca58b640 1 -- 172.21.15.167:0/1614592551 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c41a3bd0 con 0x7fa8c4100490 2024-06-18T13:03:23.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.789+0000 7fa8ca58b640 1 -- 172.21.15.167:0/1614592551 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa8c40fced0 con 0x7fa8c4100490 2024-06-18T13:03:23.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.790+0000 7fa8c17fa640 1 -- 172.21.15.167:0/1614592551 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7fa8b4042070 con 0x7fa8c4100490 2024-06-18T13:03:23.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.790+0000 7fa8c17fa640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fa8ac06e040 0x7fa8ac070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:23.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.790+0000 7fa8c17fa640 1 -- 172.21.15.167:0/1614592551 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7fa8b40aed90 con 0x7fa8c4100490 2024-06-18T13:03:23.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.790+0000 7fa8c37fe640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fa8ac06e040 0x7fa8ac070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:23.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.791+0000 7fa8c17fa640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696] conn(0x7fa8ac077e30 0x7fa8ac07a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:23.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.791+0000 7fa8c17fa640 1 -- 172.21.15.167:0/1614592551 --> [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8ac07a940 con 0x7fa8ac077e30 2024-06-18T13:03:23.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.791+0000 7fa8c17fa640 1 -- 172.21.15.167:0/1614592551 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fa8b40b1490 con 0x7fa8c4100490 2024-06-18T13:03:23.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.791+0000 7fa8c37fe640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fa8ac06e040 0x7fa8ac070500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80095f0 tx=0x7fa8b8005f90 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:23.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.791+0000 7fa8c8b28640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696] conn(0x7fa8ac077e30 0x7fa8ac07a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:23.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.791+0000 7fa8c8b28640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696] conn(0x7fa8ac077e30 0x7fa8ac07a270 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:23.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.792+0000 7fa8c17fa640 1 -- 172.21.15.167:0/1614592551 <== osd.4 v2:172.21.15.179:6800/3925771696 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa8ac07a940 con 0x7fa8ac077e30 2024-06-18T13:03:23.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.806+0000 7fa8ca58b640 1 -- 172.21.15.167:0/1614592551 --> [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa8c4101970 con 0x7fa8ac077e30 2024-06-18T13:03:23.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.807+0000 7fa8c17fa640 1 -- 172.21.15.167:0/1614592551 <== osd.4 v2:172.21.15.179:6800/3925771696 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa8c4101970 con 0x7fa8ac077e30 2024-06-18T13:03:23.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.807+0000 7fa8ca58b640 1 -- 172.21.15.167:0/1614592551 >> [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696] conn(0x7fa8ac077e30 msgr2=0x7fa8ac07a270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:23.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.807+0000 7fa8ca58b640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.179:6800/3925771696,v1:172.21.15.179:6801/3925771696] conn(0x7fa8ac077e30 0x7fa8ac07a270 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:23.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.808+0000 7fa8ca58b640 1 -- 172.21.15.167:0/1614592551 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fa8ac06e040 msgr2=0x7fa8ac070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:23.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.808+0000 7fa8ca58b640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fa8ac06e040 0x7fa8ac070500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80095f0 tx=0x7fa8b8005f90 comp rx=0 tx=0).stop 2024-06-18T13:03:23.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.808+0000 7fa8ca58b640 1 -- 172.21.15.167:0/1614592551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8c4100490 msgr2=0x7fa8c4197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:23.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.808+0000 7fa8ca58b640 1 --2- 172.21.15.167:0/1614592551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8c4100490 0x7fa8c4197ff0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4005940 tx=0x7fa8b4005e50 comp rx=0 tx=0).stop 2024-06-18T13:03:23.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.808+0000 7fa8c8b28640 1 -- 172.21.15.167:0/1614592551 reap_dead start 2024-06-18T13:03:23.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.808+0000 7fa8ca58b640 1 -- 172.21.15.167:0/1614592551 shutdown_connections 2024-06-18T13:03:23.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.808+0000 7fa8ca58b640 1 -- 172.21.15.167:0/1614592551 >> 172.21.15.167:0/1614592551 conn(0x7fa8c40fbf50 msgr2=0x7fa8c40fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:23.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.808+0000 7fa8ca58b640 1 -- 172.21.15.167:0/1614592551 shutdown_connections 2024-06-18T13:03:23.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:23.808+0000 7fa8ca58b640 1 -- 172.21.15.167:0/1614592551 wait complete. 2024-06-18T13:03:24.553 INFO:teuthology.orchestra.run.smithi167.stdout:34359738416 2024-06-18T13:03:24.554 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-18T13:03:24.704 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:24 smithi179 ceph-mon[33312]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-18T13:03:24.704 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:24 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:24.704 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:24 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:24 smithi167 ceph-mon[37701]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-18T13:03:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:24 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:24 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:24 smithi167 ceph-mon[29496]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-18T13:03:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:24 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:24 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:24.985 INFO:teuthology.orchestra.run.smithi167.stdout:133143986206 2024-06-18T13:03:24.986 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-18T13:03:25.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:03:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:03:26.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.344+0000 7fd51f3a6640 1 -- 172.21.15.167:0/1784931335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd518101e50 msgr2=0x7fd518102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:26.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.344+0000 7fd51f3a6640 1 --2- 172.21.15.167:0/1784931335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd518101e50 0x7fd518102290 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd508009930 tx=0x7fd50802a420 comp rx=0 tx=0).stop 2024-06-18T13:03:26.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.344+0000 7fd51f3a6640 1 -- 172.21.15.167:0/1784931335 shutdown_connections 2024-06-18T13:03:26.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.344+0000 7fd51f3a6640 1 --2- 172.21.15.167:0/1784931335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5181027d0 0x7fd51810d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:26.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.344+0000 7fd51f3a6640 1 --2- 172.21.15.167:0/1784931335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd518101e50 0x7fd518102290 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:26.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.344+0000 7fd51f3a6640 1 --2- 172.21.15.167:0/1784931335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd518100560 0x7fd518100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:26.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.344+0000 7fd51f3a6640 1 -- 172.21.15.167:0/1784931335 >> 172.21.15.167:0/1784931335 conn(0x7fd5180fbf80 msgr2=0x7fd5180fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:26.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.344+0000 7fd51f3a6640 1 -- 172.21.15.167:0/1784931335 shutdown_connections 2024-06-18T13:03:26.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.344+0000 7fd51f3a6640 1 -- 172.21.15.167:0/1784931335 wait complete. 2024-06-18T13:03:26.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51f3a6640 1 Processor -- start 2024-06-18T13:03:26.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51f3a6640 1 -- start start 2024-06-18T13:03:26.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51f3a6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd518100560 0x7fd5181961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:26.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51f3a6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd518101e50 0x7fd5181966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:26.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51f3a6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5181027d0 0x7fd51819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:26.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51f3a6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd51810fbc0 con 0x7fd5181027d0 2024-06-18T13:03:26.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51f3a6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd51810fa40 con 0x7fd518101e50 2024-06-18T13:03:26.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51f3a6640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd51810fd40 con 0x7fd518100560 2024-06-18T13:03:26.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51d142640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd518100560 0x7fd5181961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:26.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51d142640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd518100560 0x7fd5181961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:32892/0 (socket says 172.21.15.167:32892) 2024-06-18T13:03:26.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51d142640 1 -- 172.21.15.167:0/2690433398 learned_addr learned my addr 172.21.15.167:0/2690433398 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:26.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51d943640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5181027d0 0x7fd51819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:26.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51c941640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd518101e50 0x7fd5181966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:26.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51d142640 1 -- 172.21.15.167:0/2690433398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd518101e50 msgr2=0x7fd5181966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:26.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51d142640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd518101e50 0x7fd5181966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:26.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51d142640 1 -- 172.21.15.167:0/2690433398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5181027d0 msgr2=0x7fd51819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:26.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51d142640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5181027d0 0x7fd51819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:26.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51d142640 1 -- 172.21.15.167:0/2690433398 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51819b150 con 0x7fd518100560 2024-06-18T13:03:26.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51d943640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5181027d0 0x7fd51819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:03:26.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.345+0000 7fd51c941640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd518101e50 0x7fd5181966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T13:03:26.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.346+0000 7fd51d142640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd518100560 0x7fd5181961a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd51400ba50 tx=0x7fd51400be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:26.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.346+0000 7fd50e7fc640 1 -- 172.21.15.167:0/2690433398 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514019070 con 0x7fd518100560 2024-06-18T13:03:26.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.346+0000 7fd50e7fc640 1 -- 172.21.15.167:0/2690433398 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fd514014440 con 0x7fd518100560 2024-06-18T13:03:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.346+0000 7fd51f3a6640 1 -- 172.21.15.167:0/2690433398 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51819b440 con 0x7fd518100560 2024-06-18T13:03:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.346+0000 7fd50e7fc640 1 -- 172.21.15.167:0/2690433398 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5140075d0 con 0x7fd518100560 2024-06-18T13:03:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.346+0000 7fd51f3a6640 1 -- 172.21.15.167:0/2690433398 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5181a2d20 con 0x7fd518100560 2024-06-18T13:03:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.347+0000 7fd51f3a6640 1 -- 172.21.15.167:0/2690433398 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5180fcd40 con 0x7fd518100560 2024-06-18T13:03:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.347+0000 7fd50e7fc640 1 -- 172.21.15.167:0/2690433398 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7fd51401c070 con 0x7fd518100560 2024-06-18T13:03:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.348+0000 7fd50e7fc640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fd50406e040 0x7fd504070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.348+0000 7fd50e7fc640 1 -- 172.21.15.167:0/2690433398 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7fd51408dd10 con 0x7fd518100560 2024-06-18T13:03:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.348+0000 7fd51c941640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fd50406e040 0x7fd504070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.348+0000 7fd50e7fc640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065] conn(0x7fd504077e30 0x7fd50407a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.348+0000 7fd50e7fc640 1 -- 172.21.15.167:0/2690433398 --> [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd514018ec0 con 0x7fd504077e30 2024-06-18T13:03:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.348+0000 7fd50e7fc640 1 -- 172.21.15.167:0/2690433398 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fd514091020 con 0x7fd518100560 2024-06-18T13:03:26.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.348+0000 7fd51c941640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fd50406e040 0x7fd504070500 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd508000c00 tx=0x7fd508009610 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:26.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.348+0000 7fd51d943640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065] conn(0x7fd504077e30 0x7fd50407a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:26.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.349+0000 7fd51d943640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065] conn(0x7fd504077e30 0x7fd50407a270 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:26.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.349+0000 7fd50e7fc640 1 -- 172.21.15.167:0/2690433398 <== osd.6 v2:172.21.15.179:6816/153587065 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd514018ec0 con 0x7fd504077e30 2024-06-18T13:03:26.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.363+0000 7fd51f3a6640 1 -- 172.21.15.167:0/2690433398 --> [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd518101960 con 0x7fd504077e30 2024-06-18T13:03:26.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.364+0000 7fd50e7fc640 1 -- 172.21.15.167:0/2690433398 <== osd.6 v2:172.21.15.179:6816/153587065 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd518101960 con 0x7fd504077e30 2024-06-18T13:03:26.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.364+0000 7fd51f3a6640 1 -- 172.21.15.167:0/2690433398 >> [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065] conn(0x7fd504077e30 msgr2=0x7fd50407a270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:26.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.364+0000 7fd51f3a6640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.179:6816/153587065,v1:172.21.15.179:6817/153587065] conn(0x7fd504077e30 0x7fd50407a270 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:26.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.364+0000 7fd51f3a6640 1 -- 172.21.15.167:0/2690433398 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fd50406e040 msgr2=0x7fd504070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:26.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.364+0000 7fd51f3a6640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fd50406e040 0x7fd504070500 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd508000c00 tx=0x7fd508009610 comp rx=0 tx=0).stop 2024-06-18T13:03:26.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.364+0000 7fd51f3a6640 1 -- 172.21.15.167:0/2690433398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd518100560 msgr2=0x7fd5181961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:26.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.364+0000 7fd51f3a6640 1 --2- 172.21.15.167:0/2690433398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd518100560 0x7fd5181961a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd51400ba50 tx=0x7fd51400be50 comp rx=0 tx=0).stop 2024-06-18T13:03:26.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.364+0000 7fd51d943640 1 -- 172.21.15.167:0/2690433398 reap_dead start 2024-06-18T13:03:26.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.364+0000 7fd51f3a6640 1 -- 172.21.15.167:0/2690433398 shutdown_connections 2024-06-18T13:03:26.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.364+0000 7fd51f3a6640 1 -- 172.21.15.167:0/2690433398 >> 172.21.15.167:0/2690433398 conn(0x7fd5180fbf80 msgr2=0x7fd5180fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:26.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.364+0000 7fd51f3a6640 1 -- 172.21.15.167:0/2690433398 shutdown_connections 2024-06-18T13:03:26.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:26.364+0000 7fd51f3a6640 1 -- 172.21.15.167:0/2690433398 wait complete. 2024-06-18T13:03:26.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:26 smithi179 ceph-mon[33312]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-18T13:03:26.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:26 smithi167 ceph-mon[37701]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-18T13:03:26.924 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:26 smithi167 ceph-mon[29496]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-18T13:03:26.951 INFO:teuthology.orchestra.run.smithi167.stdout:201863462933 2024-06-18T13:03:26.952 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-18T13:03:27.023 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:27.024 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:27.098 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:27.099 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:27.099 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:28.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: Adjusting osd_memory_target on smithi167 to 3760M 2024-06-18T13:03:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:03:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:03:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: Updating smithi167:/etc/ceph/ceph.conf 2024-06-18T13:03:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-18T13:03:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: Updating smithi167:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T13:03:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: Updating smithi179:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T13:03:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-06-18T13:03:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-06-18T13:03:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: Updating smithi167:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-18T13:03:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: Updating smithi179:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-18T13:03:28.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:28.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:28.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:28.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:28.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:28.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: Adjusting osd_memory_target on smithi167 to 3760M 2024-06-18T13:03:28.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:03:28.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:03:28.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: Updating smithi167:/etc/ceph/ceph.conf 2024-06-18T13:03:28.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-18T13:03:28.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: Updating smithi167:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T13:03:28.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: Updating smithi179:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T13:03:28.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-06-18T13:03:28.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-06-18T13:03:28.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: Updating smithi167:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-18T13:03:28.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: Updating smithi179:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-18T13:03:28.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:28 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:03:28.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:03:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: Adjusting osd_memory_target on smithi167 to 3760M 2024-06-18T13:03:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:03:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:03:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: Updating smithi167:/etc/ceph/ceph.conf 2024-06-18T13:03:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-18T13:03:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: Updating smithi167:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T13:03:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: Updating smithi179:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-18T13:03:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-06-18T13:03:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-06-18T13:03:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: Updating smithi167:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-18T13:03:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: Updating smithi179:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-18T13:03:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:28 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:29.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:29 smithi167 ceph-mon[29496]: Deploying daemon alertmanager.a on smithi167 2024-06-18T13:03:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:29 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:03:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:29 smithi167 ceph-mon[37701]: Deploying daemon alertmanager.a on smithi167 2024-06-18T13:03:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:29 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:03:29.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:29 smithi179 ceph-mon[33312]: Deploying daemon alertmanager.a on smithi167 2024-06-18T13:03:29.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:29 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:03:30.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:30 smithi167 ceph-mon[37701]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:30 smithi167 ceph-mon[29496]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:30.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.828+0000 7f13cf675640 1 -- 172.21.15.167:0/2111187682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13c81066c0 msgr2=0x7f13c810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:30.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.828+0000 7f13cf675640 1 --2- 172.21.15.167:0/2111187682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13c81066c0 0x7f13c810e240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009be0 tx=0x7f13c002a6c0 comp rx=0 tx=0).stop 2024-06-18T13:03:30.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.828+0000 7f13cf675640 1 -- 172.21.15.167:0/2111187682 shutdown_connections 2024-06-18T13:03:30.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.828+0000 7f13cf675640 1 --2- 172.21.15.167:0/2111187682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13c81066c0 0x7f13c810e240 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:30.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.828+0000 7f13cf675640 1 --2- 172.21.15.167:0/2111187682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c8101f40 0x7f13c8105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:30.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.828+0000 7f13cf675640 1 --2- 172.21.15.167:0/2111187682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13c8101590 0x7f13c8101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:30.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.828+0000 7f13cf675640 1 -- 172.21.15.167:0/2111187682 >> 172.21.15.167:0/2111187682 conn(0x7f13c80fd070 msgr2=0x7f13c80ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:30.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.828+0000 7f13cf675640 1 -- 172.21.15.167:0/2111187682 shutdown_connections 2024-06-18T13:03:30.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.828+0000 7f13cf675640 1 -- 172.21.15.167:0/2111187682 wait complete. 2024-06-18T13:03:30.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cf675640 1 Processor -- start 2024-06-18T13:03:30.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cf675640 1 -- start start 2024-06-18T13:03:30.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cf675640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13c8101590 0x7f13c81971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:30.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cf675640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c8101f40 0x7f13c8197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:30.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cf675640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13c81066c0 0x7f13c819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:30.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cf675640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8110bf0 con 0x7f13c8101590 2024-06-18T13:03:30.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cf675640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8110a70 con 0x7f13c8101f40 2024-06-18T13:03:30.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cf675640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f13c8110d70 con 0x7f13c81066c0 2024-06-18T13:03:30.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cd411640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13c8101590 0x7f13c81971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cdc12640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13c81066c0 0x7f13c819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cd411640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13c8101590 0x7f13c81971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53752/0 (socket says 172.21.15.167:53752) 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cd411640 1 -- 172.21.15.167:0/1157493468 learned_addr learned my addr 172.21.15.167:0/1157493468 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13ccc10640 1 --2- 172.21.15.167:0/1157493468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c8101f40 0x7f13c8197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cd411640 1 -- 172.21.15.167:0/1157493468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13c81066c0 msgr2=0x7f13c819ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cd411640 1 --2- 172.21.15.167:0/1157493468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13c81066c0 0x7f13c819ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cd411640 1 -- 172.21.15.167:0/1157493468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c8101f40 msgr2=0x7f13c8197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cd411640 1 --2- 172.21.15.167:0/1157493468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c8101f40 0x7f13c8197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cd411640 1 -- 172.21.15.167:0/1157493468 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c819c170 con 0x7f13c8101590 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13ccc10640 1 --2- 172.21.15.167:0/1157493468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c8101f40 0x7f13c8197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.829+0000 7f13cd411640 1 --2- 172.21.15.167:0/1157493468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13c8101590 0x7f13c81971c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f13c400e9c0 tx=0x7f13c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.830+0000 7f13ba7fc640 1 -- 172.21.15.167:0/1157493468 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c400cde0 con 0x7f13c8101590 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.830+0000 7f13cf675640 1 -- 172.21.15.167:0/1157493468 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c819c460 con 0x7f13c8101590 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.830+0000 7f13ba7fc640 1 -- 172.21.15.167:0/1157493468 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f13c40035f0 con 0x7f13c8101590 2024-06-18T13:03:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.830+0000 7f13ba7fc640 1 -- 172.21.15.167:0/1157493468 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c40044a0 con 0x7f13c8101590 2024-06-18T13:03:30.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.830+0000 7f13cf675640 1 -- 172.21.15.167:0/1157493468 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c81a3d40 con 0x7f13c8101590 2024-06-18T13:03:30.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.831+0000 7f13cf675640 1 -- 172.21.15.167:0/1157493468 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c8101970 con 0x7f13c8101590 2024-06-18T13:03:30.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.831+0000 7f13ba7fc640 1 -- 172.21.15.167:0/1157493468 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f13c4004000 con 0x7f13c8101590 2024-06-18T13:03:30.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.832+0000 7f13ba7fc640 1 --2- 172.21.15.167:0/1157493468 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f13b406e040 0x7f13b4070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:30.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.832+0000 7f13ba7fc640 1 -- 172.21.15.167:0/1157493468 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7f13c408e9c0 con 0x7f13c8101590 2024-06-18T13:03:30.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.832+0000 7f13ccc10640 1 --2- 172.21.15.167:0/1157493468 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f13b406e040 0x7f13b4070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:30.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.832+0000 7f13ccc10640 1 --2- 172.21.15.167:0/1157493468 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f13b406e040 0x7f13b4070500 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f13bc006fd0 tx=0x7f13bc008040 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:30.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:30.836+0000 7f13ba7fc640 1 -- 172.21.15.167:0/1157493468 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f13c4090e20 con 0x7f13c8101590 2024-06-18T13:03:30.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:30 smithi179 ceph-mon[33312]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:31.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:31.018+0000 7f13cf675640 1 -- 172.21.15.167:0/1157493468 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f13c81a3f70 con 0x7f13c8101590 2024-06-18T13:03:31.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:31.018+0000 7f13ba7fc640 1 -- 172.21.15.167:0/1157493468 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f13c405bdc0 con 0x7f13c8101590 2024-06-18T13:03:31.018 INFO:teuthology.orchestra.run.smithi167.stdout:167503724571 2024-06-18T13:03:31.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:31.020+0000 7f13cf675640 1 -- 172.21.15.167:0/1157493468 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f13b406e040 msgr2=0x7f13b4070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:31.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:31.020+0000 7f13cf675640 1 --2- 172.21.15.167:0/1157493468 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f13b406e040 0x7f13b4070500 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f13bc006fd0 tx=0x7f13bc008040 comp rx=0 tx=0).stop 2024-06-18T13:03:31.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:31.020+0000 7f13cf675640 1 -- 172.21.15.167:0/1157493468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13c8101590 msgr2=0x7f13c81971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:31.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:31.020+0000 7f13cf675640 1 --2- 172.21.15.167:0/1157493468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13c8101590 0x7f13c81971c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f13c400e9c0 tx=0x7f13c400ee80 comp rx=0 tx=0).stop 2024-06-18T13:03:31.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:31.020+0000 7f13cf675640 1 -- 172.21.15.167:0/1157493468 shutdown_connections 2024-06-18T13:03:31.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:31.020+0000 7f13cf675640 1 --2- 172.21.15.167:0/1157493468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13c81066c0 0x7f13c819ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:31.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:31.020+0000 7f13cf675640 1 --2- 172.21.15.167:0/1157493468 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f13b406e040 0x7f13b4070500 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:31.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:31.020+0000 7f13cf675640 1 --2- 172.21.15.167:0/1157493468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c8101f40 0x7f13c8197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:31.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:31.020+0000 7f13cf675640 1 --2- 172.21.15.167:0/1157493468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13c8101590 0x7f13c81971c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:31.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:31.020+0000 7f13cf675640 1 -- 172.21.15.167:0/1157493468 >> 172.21.15.167:0/1157493468 conn(0x7f13c80fd070 msgr2=0x7f13c80fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:31.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:31.020+0000 7f13cf675640 1 -- 172.21.15.167:0/1157493468 shutdown_connections 2024-06-18T13:03:31.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:31.020+0000 7f13cf675640 1 -- 172.21.15.167:0/1157493468 wait complete. 2024-06-18T13:03:31.709 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1157493468' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-18T13:03:31.709 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1157493468' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-18T13:03:31.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1157493468' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-18T13:03:32.700 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:32 smithi167 ceph-mon[37701]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:03:32.700 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:32 smithi167 ceph-mon[29496]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:03:32.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:32 smithi179 ceph-mon[33312]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:03:33.314 INFO:teuthology.orchestra.run.smithi167.stdout:77309411366 2024-06-18T13:03:33.315 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-18T13:03:33.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.526+0000 7f45bd5ea640 1 -- 172.21.15.167:0/2182595326 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45b81055c0 msgr2=0x7f45b810d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:33.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.526+0000 7f45bd5ea640 1 --2- 172.21.15.167:0/2182595326 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45b81055c0 0x7f45b810d140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009ca0 tx=0x7f45b002a8c0 comp rx=0 tx=0).stop 2024-06-18T13:03:33.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.526+0000 7f45bd5ea640 1 -- 172.21.15.167:0/2182595326 shutdown_connections 2024-06-18T13:03:33.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.526+0000 7f45bd5ea640 1 --2- 172.21.15.167:0/2182595326 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45b81055c0 0x7f45b810d140 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:33.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.526+0000 7f45bd5ea640 1 --2- 172.21.15.167:0/2182595326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b8100e40 0x7f45b8104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:33.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.526+0000 7f45bd5ea640 1 --2- 172.21.15.167:0/2182595326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b8100490 0x7f45b8100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:33.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.526+0000 7f45bd5ea640 1 -- 172.21.15.167:0/2182595326 >> 172.21.15.167:0/2182595326 conn(0x7f45b80fbf50 msgr2=0x7f45b80fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:33.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.526+0000 7f45bd5ea640 1 -- 172.21.15.167:0/2182595326 shutdown_connections 2024-06-18T13:03:33.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.526+0000 7f45bd5ea640 1 -- 172.21.15.167:0/2182595326 wait complete. 2024-06-18T13:03:33.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.526+0000 7f45bd5ea640 1 Processor -- start 2024-06-18T13:03:33.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45bd5ea640 1 -- start start 2024-06-18T13:03:33.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45bd5ea640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b8100490 0x7f45b8195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:33.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45bd5ea640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45b8100e40 0x7f45b8196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:33.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45bd5ea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b81055c0 0x7f45b8196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:33.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45bd5ea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f45b810faf0 con 0x7f45b81055c0 2024-06-18T13:03:33.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45bd5ea640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f45b810f970 con 0x7f45b8100490 2024-06-18T13:03:33.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45bd5ea640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f45b810fc70 con 0x7f45b8100e40 2024-06-18T13:03:33.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45b67fc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45b8100e40 0x7f45b8196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:33.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45b67fc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45b8100e40 0x7f45b8196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:38722/0 (socket says 172.21.15.167:38722) 2024-06-18T13:03:33.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45b67fc640 1 -- 172.21.15.167:0/3875298765 learned_addr learned my addr 172.21.15.167:0/3875298765 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:33.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45b77fe640 1 --2- 172.21.15.167:0/3875298765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b81055c0 0x7f45b8196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:33.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45b6ffd640 1 --2- 172.21.15.167:0/3875298765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b8100490 0x7f45b8195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:33.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45b67fc640 1 -- 172.21.15.167:0/3875298765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b8100490 msgr2=0x7f45b8195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:33.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45b67fc640 1 --2- 172.21.15.167:0/3875298765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b8100490 0x7f45b8195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:33.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45b67fc640 1 -- 172.21.15.167:0/3875298765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b81055c0 msgr2=0x7f45b8196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:33.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45b67fc640 1 --2- 172.21.15.167:0/3875298765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b81055c0 0x7f45b8196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:33.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45b67fc640 1 -- 172.21.15.167:0/3875298765 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b81a5630 con 0x7f45b8100e40 2024-06-18T13:03:33.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.527+0000 7f45b67fc640 1 --2- 172.21.15.167:0/3875298765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45b8100e40 0x7f45b8196300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00b4a0 tx=0x7f45ac00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:33.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.528+0000 7f459ffff640 1 -- 172.21.15.167:0/3875298765 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac014070 con 0x7f45b8100e40 2024-06-18T13:03:33.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.528+0000 7f459ffff640 1 -- 172.21.15.167:0/3875298765 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f45ac010a90 con 0x7f45b8100e40 2024-06-18T13:03:33.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.528+0000 7f45bd5ea640 1 -- 172.21.15.167:0/3875298765 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b81a5920 con 0x7f45b8100e40 2024-06-18T13:03:33.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.529+0000 7f459ffff640 1 -- 172.21.15.167:0/3875298765 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac017b40 con 0x7f45b8100e40 2024-06-18T13:03:33.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.529+0000 7f45bd5ea640 1 -- 172.21.15.167:0/3875298765 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b81a5e60 con 0x7f45b8100e40 2024-06-18T13:03:33.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.529+0000 7f45bd5ea640 1 -- 172.21.15.167:0/3875298765 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b8101970 con 0x7f45b8100e40 2024-06-18T13:03:33.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.530+0000 7f459ffff640 1 -- 172.21.15.167:0/3875298765 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f45ac017070 con 0x7f45b8100e40 2024-06-18T13:03:33.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.530+0000 7f459ffff640 1 --2- 172.21.15.167:0/3875298765 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f45a006df10 0x7f45a00703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:33.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.530+0000 7f45b6ffd640 1 --2- 172.21.15.167:0/3875298765 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f45a006df10 0x7f45a00703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:33.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.530+0000 7f459ffff640 1 -- 172.21.15.167:0/3875298765 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7f45ac08d550 con 0x7f45b8100e40 2024-06-18T13:03:33.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.531+0000 7f45b6ffd640 1 --2- 172.21.15.167:0/3875298765 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f45a006df10 0x7f45a00703d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f45a80095f0 tx=0x7f45a8005f90 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:33.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.535+0000 7f459ffff640 1 -- 172.21.15.167:0/3875298765 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f45ac010c40 con 0x7f45b8100e40 2024-06-18T13:03:33.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.730+0000 7f45bd5ea640 1 -- 172.21.15.167:0/3875298765 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f45b806a530 con 0x7f45b8100e40 2024-06-18T13:03:33.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.731+0000 7f459ffff640 1 -- 172.21.15.167:0/3875298765 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f45ac05e600 con 0x7f45b8100e40 2024-06-18T13:03:33.731 INFO:teuthology.orchestra.run.smithi167.stdout:34359738419 2024-06-18T13:03:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.733+0000 7f45bd5ea640 1 -- 172.21.15.167:0/3875298765 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f45a006df10 msgr2=0x7f45a00703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.733+0000 7f45bd5ea640 1 --2- 172.21.15.167:0/3875298765 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f45a006df10 0x7f45a00703d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f45a80095f0 tx=0x7f45a8005f90 comp rx=0 tx=0).stop 2024-06-18T13:03:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.733+0000 7f45bd5ea640 1 -- 172.21.15.167:0/3875298765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45b8100e40 msgr2=0x7f45b8196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.733+0000 7f45bd5ea640 1 --2- 172.21.15.167:0/3875298765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45b8100e40 0x7f45b8196300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00b4a0 tx=0x7f45ac00b960 comp rx=0 tx=0).stop 2024-06-18T13:03:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.733+0000 7f45bd5ea640 1 -- 172.21.15.167:0/3875298765 shutdown_connections 2024-06-18T13:03:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.733+0000 7f45bd5ea640 1 --2- 172.21.15.167:0/3875298765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b81055c0 0x7f45b8196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.733+0000 7f45bd5ea640 1 --2- 172.21.15.167:0/3875298765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45b8100e40 0x7f45b8196300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.733+0000 7f45bd5ea640 1 --2- 172.21.15.167:0/3875298765 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f45a006df10 0x7f45a00703d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.733+0000 7f45bd5ea640 1 --2- 172.21.15.167:0/3875298765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b8100490 0x7f45b8195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.733+0000 7f45bd5ea640 1 -- 172.21.15.167:0/3875298765 >> 172.21.15.167:0/3875298765 conn(0x7f45b80fbf50 msgr2=0x7f45b80fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.733+0000 7f45bd5ea640 1 -- 172.21.15.167:0/3875298765 shutdown_connections 2024-06-18T13:03:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:33.733+0000 7f45bd5ea640 1 -- 172.21.15.167:0/3875298765 wait complete. 2024-06-18T13:03:34.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.329+0000 7fed47d23640 1 -- 172.21.15.167:0/4232054895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed40101e50 msgr2=0x7fed40102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:34.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.329+0000 7fed47d23640 1 --2- 172.21.15.167:0/4232054895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed40101e50 0x7fed40102290 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fed34009930 tx=0x7fed3402a420 comp rx=0 tx=0).stop 2024-06-18T13:03:34.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.329+0000 7fed47d23640 1 -- 172.21.15.167:0/4232054895 shutdown_connections 2024-06-18T13:03:34.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.329+0000 7fed47d23640 1 --2- 172.21.15.167:0/4232054895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed401027d0 0x7fed4010d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:34.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.329+0000 7fed47d23640 1 --2- 172.21.15.167:0/4232054895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed40101e50 0x7fed40102290 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:34.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.329+0000 7fed47d23640 1 --2- 172.21.15.167:0/4232054895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed40100560 0x7fed40100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:34.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.329+0000 7fed47d23640 1 -- 172.21.15.167:0/4232054895 >> 172.21.15.167:0/4232054895 conn(0x7fed400fbf80 msgr2=0x7fed400fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:34.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.330+0000 7fed47d23640 1 -- 172.21.15.167:0/4232054895 shutdown_connections 2024-06-18T13:03:34.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.330+0000 7fed47d23640 1 -- 172.21.15.167:0/4232054895 wait complete. 2024-06-18T13:03:34.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.330+0000 7fed47d23640 1 Processor -- start 2024-06-18T13:03:34.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.330+0000 7fed47d23640 1 -- start start 2024-06-18T13:03:34.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.330+0000 7fed47d23640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed40100560 0x7fed40198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:34.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.330+0000 7fed47d23640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed40101e50 0x7fed40198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:34.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.330+0000 7fed47d23640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed401027d0 0x7fed401a3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:34.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.330+0000 7fed47d23640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed4010fbc0 con 0x7fed40100560 2024-06-18T13:03:34.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.330+0000 7fed47d23640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fed4010fa40 con 0x7fed401027d0 2024-06-18T13:03:34.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.330+0000 7fed47d23640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fed4010fd40 con 0x7fed40101e50 2024-06-18T13:03:34.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.330+0000 7fed45abf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed40100560 0x7fed40198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:34.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.330+0000 7fed452be640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed40101e50 0x7fed40198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:34.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed452be640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed40101e50 0x7fed40198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:38730/0 (socket says 172.21.15.167:38730) 2024-06-18T13:03:34.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed452be640 1 -- 172.21.15.167:0/3983000065 learned_addr learned my addr 172.21.15.167:0/3983000065 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:34.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed45abf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed40100560 0x7fed40198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56050/0 (socket says 172.21.15.167:56050) 2024-06-18T13:03:34.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed462c0640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed401027d0 0x7fed401a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:34.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed452be640 1 -- 172.21.15.167:0/3983000065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed401027d0 msgr2=0x7fed401a3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:34.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed452be640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed401027d0 0x7fed401a3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:34.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed452be640 1 -- 172.21.15.167:0/3983000065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed40100560 msgr2=0x7fed40198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:34.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed452be640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed40100560 0x7fed40198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:34.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed452be640 1 -- 172.21.15.167:0/3983000065 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed401a36d0 con 0x7fed40101e50 2024-06-18T13:03:34.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed462c0640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed401027d0 0x7fed401a3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-18T13:03:34.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed45abf640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed40100560 0x7fed40198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:03:34.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed452be640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed40101e50 0x7fed40198650 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fed34005940 tx=0x7fed34004f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:34.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed3effd640 1 -- 172.21.15.167:0/3983000065 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3403f070 con 0x7fed40101e50 2024-06-18T13:03:34.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed47d23640 1 -- 172.21.15.167:0/3983000065 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed401a38a0 con 0x7fed40101e50 2024-06-18T13:03:34.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed3effd640 1 -- 172.21.15.167:0/3983000065 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fed34005590 con 0x7fed40101e50 2024-06-18T13:03:34.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed3effd640 1 -- 172.21.15.167:0/3983000065 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34035030 con 0x7fed40101e50 2024-06-18T13:03:34.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.331+0000 7fed47d23640 1 -- 172.21.15.167:0/3983000065 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fed401a3d50 con 0x7fed40101e50 2024-06-18T13:03:34.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.332+0000 7fed47d23640 1 -- 172.21.15.167:0/3983000065 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed40100940 con 0x7fed40101e50 2024-06-18T13:03:34.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.333+0000 7fed3effd640 1 -- 172.21.15.167:0/3983000065 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7fed34042070 con 0x7fed40101e50 2024-06-18T13:03:34.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.333+0000 7fed3effd640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fed2806e040 0x7fed28070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:34.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.333+0000 7fed3effd640 1 -- 172.21.15.167:0/3983000065 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7fed340aed00 con 0x7fed40101e50 2024-06-18T13:03:34.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.333+0000 7fed45abf640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fed2806e040 0x7fed28070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:34.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.334+0000 7fed45abf640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fed2806e040 0x7fed28070500 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fed3000a7b0 tx=0x7fed3000d040 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:34.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.337+0000 7fed3effd640 1 -- 172.21.15.167:0/3983000065 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fed3407fdb0 con 0x7fed40101e50 2024-06-18T13:03:34.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.527+0000 7fed47d23640 1 -- 172.21.15.167:0/3983000065 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fed401a3f80 con 0x7fed40101e50 2024-06-18T13:03:34.527 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:34 smithi167 ceph-mon[29496]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:03:34.527 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3875298765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-18T13:03:34.528 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:34 smithi167 ceph-mon[37701]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:03:34.528 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3875298765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-18T13:03:34.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.527+0000 7fed3effd640 1 -- 172.21.15.167:0/3983000065 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fed340845e0 con 0x7fed40101e50 2024-06-18T13:03:34.528 INFO:teuthology.orchestra.run.smithi167.stdout:133143986208 2024-06-18T13:03:34.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.530+0000 7fed47d23640 1 -- 172.21.15.167:0/3983000065 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fed2806e040 msgr2=0x7fed28070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:34.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.530+0000 7fed47d23640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fed2806e040 0x7fed28070500 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fed3000a7b0 tx=0x7fed3000d040 comp rx=0 tx=0).stop 2024-06-18T13:03:34.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.530+0000 7fed47d23640 1 -- 172.21.15.167:0/3983000065 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed40101e50 msgr2=0x7fed40198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:34.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.530+0000 7fed47d23640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed40101e50 0x7fed40198650 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fed34005940 tx=0x7fed34004f00 comp rx=0 tx=0).stop 2024-06-18T13:03:34.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.530+0000 7fed47d23640 1 -- 172.21.15.167:0/3983000065 shutdown_connections 2024-06-18T13:03:34.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.530+0000 7fed47d23640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fed2806e040 0x7fed28070500 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:34.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.530+0000 7fed47d23640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed401027d0 0x7fed401a3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:34.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.530+0000 7fed47d23640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed40101e50 0x7fed40198650 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:34.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.530+0000 7fed47d23640 1 --2- 172.21.15.167:0/3983000065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed40100560 0x7fed40198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:34.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.530+0000 7fed47d23640 1 -- 172.21.15.167:0/3983000065 >> 172.21.15.167:0/3983000065 conn(0x7fed400fbf80 msgr2=0x7fed400fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:34.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.530+0000 7fed47d23640 1 -- 172.21.15.167:0/3983000065 shutdown_connections 2024-06-18T13:03:34.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:34.530+0000 7fed47d23640 1 -- 172.21.15.167:0/3983000065 wait complete. 2024-06-18T13:03:34.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:34 smithi179 ceph-mon[33312]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:03:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3875298765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-18T13:03:35.355 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:03:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:03:35.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3983000065' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-18T13:03:35.754 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3983000065' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-18T13:03:35.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3983000065' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-18T13:03:36.306 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738416 got 34359738419 for osd.0 2024-06-18T13:03:36.307 DEBUG:teuthology.parallel:result is None 2024-06-18T13:03:36.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:36 smithi167 ceph-mon[37701]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:36.534 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:36 smithi167 ceph-mon[29496]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:36.609 INFO:teuthology.orchestra.run.smithi167.stdout:111669149730 2024-06-18T13:03:36.610 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-18T13:03:36.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:36 smithi179 ceph-mon[33312]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:37.720 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986206 got 133143986208 for osd.4 2024-06-18T13:03:37.721 DEBUG:teuthology.parallel:result is None 2024-06-18T13:03:38.660 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:38 smithi167 ceph-mon[37701]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:38.660 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:38 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:03:38.660 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:38 smithi167 ceph-mon[29496]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:38.660 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:38 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:03:38.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:38 smithi179 ceph-mon[33312]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:38 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:03:38.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:03:38 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:03:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:39 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:03:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:39 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:03:39.770 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:39.810 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724569 got 167503724571 for osd.5 2024-06-18T13:03:39.811 DEBUG:teuthology.parallel:result is None 2024-06-18T13:03:39.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:39 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:03:40.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:40 smithi167 ceph-mon[37701]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:40.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:40 smithi167 ceph-mon[29496]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:40.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.846+0000 7f72653a1640 1 -- 172.21.15.167:0/226439504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72601013e0 msgr2=0x7f72601017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:40.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.846+0000 7f72653a1640 1 --2- 172.21.15.167:0/226439504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72601013e0 0x7f72601017c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7250009900 tx=0x7f725002a410 comp rx=0 tx=0).stop 2024-06-18T13:03:40.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.846+0000 7f72653a1640 1 -- 172.21.15.167:0/226439504 shutdown_connections 2024-06-18T13:03:40.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.846+0000 7f72653a1640 1 --2- 172.21.15.167:0/226439504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7260106510 0x7f726010e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:40.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f72653a1640 1 --2- 172.21.15.167:0/226439504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7260101d90 0x7f7260105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:40.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f72653a1640 1 --2- 172.21.15.167:0/226439504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72601013e0 0x7f72601017c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:40.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f72653a1640 1 -- 172.21.15.167:0/226439504 >> 172.21.15.167:0/226439504 conn(0x7f72600fcec0 msgr2=0x7f72600ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:40.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f72653a1640 1 -- 172.21.15.167:0/226439504 shutdown_connections 2024-06-18T13:03:40.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f72653a1640 1 -- 172.21.15.167:0/226439504 wait complete. 2024-06-18T13:03:40.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f72653a1640 1 Processor -- start 2024-06-18T13:03:40.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f72653a1640 1 -- start start 2024-06-18T13:03:40.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f72653a1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72601013e0 0x7f7260197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:40.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f72653a1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7260101d90 0x7f7260197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:40.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f72653a1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7260106510 0x7f726019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:40.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f72653a1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f72601109e0 con 0x7f72601013e0 2024-06-18T13:03:40.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f72653a1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7260110860 con 0x7f7260101d90 2024-06-18T13:03:40.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f72653a1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7260110b60 con 0x7f7260106510 2024-06-18T13:03:40.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f725ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72601013e0 0x7f7260197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:40.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.847+0000 7f7264ba0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7260106510 0x7f726019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:40.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f725ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72601013e0 0x7f7260197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56074/0 (socket says 172.21.15.167:56074) 2024-06-18T13:03:40.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f7264ba0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7260106510 0x7f726019b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:38748/0 (socket says 172.21.15.167:38748) 2024-06-18T13:03:40.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f725ffff640 1 -- 172.21.15.167:0/1860564532 learned_addr learned my addr 172.21.15.167:0/1860564532 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:40.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f7257fff640 1 --2- 172.21.15.167:0/1860564532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7260101d90 0x7f7260197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:40.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f725ffff640 1 -- 172.21.15.167:0/1860564532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7260106510 msgr2=0x7f726019b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:40.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f725ffff640 1 --2- 172.21.15.167:0/1860564532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7260106510 0x7f726019b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:40.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f725ffff640 1 -- 172.21.15.167:0/1860564532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7260101d90 msgr2=0x7f7260197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:40.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f725ffff640 1 --2- 172.21.15.167:0/1860564532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7260101d90 0x7f7260197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:40.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f725ffff640 1 -- 172.21.15.167:0/1860564532 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726019bfc0 con 0x7f72601013e0 2024-06-18T13:03:40.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f7264ba0640 1 --2- 172.21.15.167:0/1860564532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7260106510 0x7f726019b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:03:40.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f725ffff640 1 --2- 172.21.15.167:0/1860564532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72601013e0 0x7f7260197010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7250005ab0 tx=0x7f7250003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:40.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f725dffb640 1 -- 172.21.15.167:0/1860564532 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725003f070 con 0x7f72601013e0 2024-06-18T13:03:40.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f725dffb640 1 -- 172.21.15.167:0/1860564532 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f7250031d30 con 0x7f72601013e0 2024-06-18T13:03:40.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f72653a1640 1 -- 172.21.15.167:0/1860564532 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f726019c250 con 0x7f72601013e0 2024-06-18T13:03:40.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f725dffb640 1 -- 172.21.15.167:0/1860564532 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250035030 con 0x7f72601013e0 2024-06-18T13:03:40.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.848+0000 7f72653a1640 1 -- 172.21.15.167:0/1860564532 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72601a3b90 con 0x7f72601013e0 2024-06-18T13:03:40.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.849+0000 7f72653a1640 1 -- 172.21.15.167:0/1860564532 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72601017c0 con 0x7f72601013e0 2024-06-18T13:03:40.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.850+0000 7f725dffb640 1 -- 172.21.15.167:0/1860564532 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f7250042070 con 0x7f72601013e0 2024-06-18T13:03:40.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.850+0000 7f725dffb640 1 --2- 172.21.15.167:0/1860564532 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f724c06e040 0x7f724c070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:40.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.850+0000 7f7257fff640 1 --2- 172.21.15.167:0/1860564532 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f724c06e040 0x7f724c070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:40.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.850+0000 7f725dffb640 1 -- 172.21.15.167:0/1860564532 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7f72500aed50 con 0x7f72601013e0 2024-06-18T13:03:40.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.851+0000 7f7257fff640 1 --2- 172.21.15.167:0/1860564532 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f724c06e040 0x7f724c070500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f72600ff090 tx=0x7f7248005c60 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:40.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:40.855+0000 7f725dffb640 1 -- 172.21.15.167:0/1860564532 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f72500b2030 con 0x7f72601013e0 2024-06-18T13:03:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:40 smithi179 ceph-mon[33312]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:41.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.036+0000 7f72653a1640 1 -- 172.21.15.167:0/1860564532 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f72601a3d60 con 0x7f72601013e0 2024-06-18T13:03:41.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.036+0000 7f725dffb640 1 -- 172.21.15.167:0/1860564532 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f725007c150 con 0x7f72601013e0 2024-06-18T13:03:41.036 INFO:teuthology.orchestra.run.smithi167.stdout:240518168596 2024-06-18T13:03:41.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.038+0000 7f72653a1640 1 -- 172.21.15.167:0/1860564532 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f724c06e040 msgr2=0x7f724c070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:41.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.038+0000 7f72653a1640 1 --2- 172.21.15.167:0/1860564532 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f724c06e040 0x7f724c070500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f72600ff090 tx=0x7f7248005c60 comp rx=0 tx=0).stop 2024-06-18T13:03:41.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.038+0000 7f72653a1640 1 -- 172.21.15.167:0/1860564532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72601013e0 msgr2=0x7f7260197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:41.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.038+0000 7f72653a1640 1 --2- 172.21.15.167:0/1860564532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72601013e0 0x7f7260197010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7250005ab0 tx=0x7f7250003c80 comp rx=0 tx=0).stop 2024-06-18T13:03:41.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.039+0000 7f72653a1640 1 -- 172.21.15.167:0/1860564532 shutdown_connections 2024-06-18T13:03:41.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.039+0000 7f72653a1640 1 --2- 172.21.15.167:0/1860564532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7260106510 0x7f726019b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:41.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.039+0000 7f72653a1640 1 --2- 172.21.15.167:0/1860564532 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f724c06e040 0x7f724c070500 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:41.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.039+0000 7f72653a1640 1 --2- 172.21.15.167:0/1860564532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7260101d90 0x7f7260197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:41.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.039+0000 7f72653a1640 1 --2- 172.21.15.167:0/1860564532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72601013e0 0x7f7260197010 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:41.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.039+0000 7f72653a1640 1 -- 172.21.15.167:0/1860564532 >> 172.21.15.167:0/1860564532 conn(0x7f72600fcec0 msgr2=0x7f72600fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:41.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.039+0000 7f72653a1640 1 -- 172.21.15.167:0/1860564532 shutdown_connections 2024-06-18T13:03:41.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.039+0000 7f72653a1640 1 -- 172.21.15.167:0/1860564532 wait complete. 2024-06-18T13:03:41.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.614+0000 7f196a4ea640 1 -- 172.21.15.167:0/2759827962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1964101e50 msgr2=0x7f1964102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:41.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.614+0000 7f196a4ea640 1 --2- 172.21.15.167:0/2759827962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1964101e50 0x7f1964102290 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1958009930 tx=0x7f195802a420 comp rx=0 tx=0).stop 2024-06-18T13:03:41.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.614+0000 7f196a4ea640 1 -- 172.21.15.167:0/2759827962 shutdown_connections 2024-06-18T13:03:41.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.614+0000 7f196a4ea640 1 --2- 172.21.15.167:0/2759827962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19641027d0 0x7f196410d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:41.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.614+0000 7f196a4ea640 1 --2- 172.21.15.167:0/2759827962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1964101e50 0x7f1964102290 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:41.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.614+0000 7f196a4ea640 1 --2- 172.21.15.167:0/2759827962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1964100560 0x7f1964100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:41.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.614+0000 7f196a4ea640 1 -- 172.21.15.167:0/2759827962 >> 172.21.15.167:0/2759827962 conn(0x7f19640fbf80 msgr2=0x7f19640fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:41.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.614+0000 7f196a4ea640 1 -- 172.21.15.167:0/2759827962 shutdown_connections 2024-06-18T13:03:41.615 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1860564532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-18T13:03:41.615 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1860564532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-18T13:03:41.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.614+0000 7f196a4ea640 1 -- 172.21.15.167:0/2759827962 wait complete. 2024-06-18T13:03:41.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.615+0000 7f196a4ea640 1 Processor -- start 2024-06-18T13:03:41.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.615+0000 7f196a4ea640 1 -- start start 2024-06-18T13:03:41.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.615+0000 7f196a4ea640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1964100560 0x7f1964196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:41.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.615+0000 7f196a4ea640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1964101e50 0x7f19641965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:41.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.615+0000 7f196a4ea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19641027d0 0x7f196419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:41.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.615+0000 7f196a4ea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f196410fb60 con 0x7f19641027d0 2024-06-18T13:03:41.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.615+0000 7f196a4ea640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f196410f9e0 con 0x7f1964101e50 2024-06-18T13:03:41.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.615+0000 7f196a4ea640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f196410fce0 con 0x7f1964100560 2024-06-18T13:03:41.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.615+0000 7f1963fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1964100560 0x7f1964196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:41.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.615+0000 7f1968a87640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19641027d0 0x7f196419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:41.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.615+0000 7f1963fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1964100560 0x7f1964196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:38762/0 (socket says 172.21.15.167:38762) 2024-06-18T13:03:41.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.615+0000 7f1963fff640 1 -- 172.21.15.167:0/3974978733 learned_addr learned my addr 172.21.15.167:0/3974978733 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:41.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.616+0000 7f19637fe640 1 --2- 172.21.15.167:0/3974978733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1964101e50 0x7f19641965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:41.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.616+0000 7f1963fff640 1 -- 172.21.15.167:0/3974978733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1964101e50 msgr2=0x7f19641965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:41.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.616+0000 7f1963fff640 1 --2- 172.21.15.167:0/3974978733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1964101e50 0x7f19641965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:41.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.616+0000 7f1963fff640 1 -- 172.21.15.167:0/3974978733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19641027d0 msgr2=0x7f196419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:41.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.616+0000 7f1963fff640 1 --2- 172.21.15.167:0/3974978733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19641027d0 0x7f196419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:41.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.616+0000 7f1963fff640 1 -- 172.21.15.167:0/3974978733 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196419b040 con 0x7f1964100560 2024-06-18T13:03:41.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.616+0000 7f1968a87640 1 --2- 172.21.15.167:0/3974978733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19641027d0 0x7f196419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:03:41.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.616+0000 7f1963fff640 1 --2- 172.21.15.167:0/3974978733 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1964100560 0x7f1964196090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1954007860 tx=0x7f1954007c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:41.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.616+0000 7f19617fa640 1 -- 172.21.15.167:0/3974978733 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954019070 con 0x7f1964100560 2024-06-18T13:03:41.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.616+0000 7f196a4ea640 1 -- 172.21.15.167:0/3974978733 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196419b330 con 0x7f1964100560 2024-06-18T13:03:41.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.616+0000 7f19617fa640 1 -- 172.21.15.167:0/3974978733 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f1954007e50 con 0x7f1964100560 2024-06-18T13:03:41.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.616+0000 7f19617fa640 1 -- 172.21.15.167:0/3974978733 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954014900 con 0x7f1964100560 2024-06-18T13:03:41.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.616+0000 7f196a4ea640 1 -- 172.21.15.167:0/3974978733 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1964069720 con 0x7f1964100560 2024-06-18T13:03:41.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.617+0000 7f196a4ea640 1 -- 172.21.15.167:0/3974978733 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1964101960 con 0x7f1964100560 2024-06-18T13:03:41.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.618+0000 7f19617fa640 1 -- 172.21.15.167:0/3974978733 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f195401c070 con 0x7f1964100560 2024-06-18T13:03:41.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.618+0000 7f19617fa640 1 --2- 172.21.15.167:0/3974978733 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f194c06e040 0x7f194c070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:41.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.618+0000 7f19617fa640 1 -- 172.21.15.167:0/3974978733 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7f195408d650 con 0x7f1964100560 2024-06-18T13:03:41.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.618+0000 7f19637fe640 1 --2- 172.21.15.167:0/3974978733 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f194c06e040 0x7f194c070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:41.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.619+0000 7f19637fe640 1 --2- 172.21.15.167:0/3974978733 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f194c06e040 0x7f194c070500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1958000c00 tx=0x7f19580041c0 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:41.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.623+0000 7f19617fa640 1 -- 172.21.15.167:0/3974978733 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f195405e700 con 0x7f1964100560 2024-06-18T13:03:41.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.802+0000 7f196a4ea640 1 -- 172.21.15.167:0/3974978733 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f19641973f0 con 0x7f1964100560 2024-06-18T13:03:41.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.802+0000 7f19617fa640 1 -- 172.21.15.167:0/3974978733 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1954062f30 con 0x7f1964100560 2024-06-18T13:03:41.802 INFO:teuthology.orchestra.run.smithi167.stdout:55834574895 2024-06-18T13:03:41.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.804+0000 7f196a4ea640 1 -- 172.21.15.167:0/3974978733 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f194c06e040 msgr2=0x7f194c070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:41.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.804+0000 7f196a4ea640 1 --2- 172.21.15.167:0/3974978733 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f194c06e040 0x7f194c070500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1958000c00 tx=0x7f19580041c0 comp rx=0 tx=0).stop 2024-06-18T13:03:41.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.804+0000 7f196a4ea640 1 -- 172.21.15.167:0/3974978733 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1964100560 msgr2=0x7f1964196090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:41.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.804+0000 7f196a4ea640 1 --2- 172.21.15.167:0/3974978733 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1964100560 0x7f1964196090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1954007860 tx=0x7f1954007c60 comp rx=0 tx=0).stop 2024-06-18T13:03:41.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.804+0000 7f196a4ea640 1 -- 172.21.15.167:0/3974978733 shutdown_connections 2024-06-18T13:03:41.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.804+0000 7f196a4ea640 1 --2- 172.21.15.167:0/3974978733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19641027d0 0x7f196419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:41.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.804+0000 7f196a4ea640 1 --2- 172.21.15.167:0/3974978733 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f194c06e040 0x7f194c070500 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:41.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.805+0000 7f196a4ea640 1 --2- 172.21.15.167:0/3974978733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1964101e50 0x7f19641965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:41.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.805+0000 7f196a4ea640 1 --2- 172.21.15.167:0/3974978733 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1964100560 0x7f1964196090 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:41.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.805+0000 7f196a4ea640 1 -- 172.21.15.167:0/3974978733 >> 172.21.15.167:0/3974978733 conn(0x7f19640fbf80 msgr2=0x7f19640fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:41.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.805+0000 7f196a4ea640 1 -- 172.21.15.167:0/3974978733 shutdown_connections 2024-06-18T13:03:41.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:41.805+0000 7f196a4ea640 1 -- 172.21.15.167:0/3974978733 wait complete. 2024-06-18T13:03:41.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1860564532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-18T13:03:42.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.696+0000 7f0eef39f640 1 -- 172.21.15.167:0/3601833115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee8101590 msgr2=0x7f0ee8101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:42.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.696+0000 7f0eef39f640 1 --2- 172.21.15.167:0/3601833115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee8101590 0x7f0ee8101970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009860 tx=0x7f0ee402a340 comp rx=0 tx=0).stop 2024-06-18T13:03:42.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 -- 172.21.15.167:0/3601833115 shutdown_connections 2024-06-18T13:03:42.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 --2- 172.21.15.167:0/3601833115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee81066c0 0x7f0ee810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:42.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 --2- 172.21.15.167:0/3601833115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ee8101f40 0x7f0ee8105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:42.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 --2- 172.21.15.167:0/3601833115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee8101590 0x7f0ee8101970 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:42.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 -- 172.21.15.167:0/3601833115 >> 172.21.15.167:0/3601833115 conn(0x7f0ee80fd070 msgr2=0x7f0ee80ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:42.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 -- 172.21.15.167:0/3601833115 shutdown_connections 2024-06-18T13:03:42.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 -- 172.21.15.167:0/3601833115 wait complete. 2024-06-18T13:03:42.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 Processor -- start 2024-06-18T13:03:42.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 -- start start 2024-06-18T13:03:42.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ee8101590 0x7f0ee8073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:42.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee8101f40 0x7f0ee80741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:42.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee81066c0 0x7f0ee8074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:42.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee8110b40 con 0x7f0ee81066c0 2024-06-18T13:03:42.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81109c0 con 0x7f0ee8101590 2024-06-18T13:03:42.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.697+0000 7f0eef39f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee8110cc0 con 0x7f0ee8101f40 2024-06-18T13:03:42.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0eed93c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee81066c0 0x7f0ee8074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:42.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0eec93a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee8101f40 0x7f0ee80741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:42.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0eed93c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee81066c0 0x7f0ee8074720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56104/0 (socket says 172.21.15.167:56104) 2024-06-18T13:03:42.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0eed93c640 1 -- 172.21.15.167:0/2855925370 learned_addr learned my addr 172.21.15.167:0/2855925370 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:42.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0eed13b640 1 --2- 172.21.15.167:0/2855925370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ee8101590 0x7f0ee8073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:42.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0eed93c640 1 -- 172.21.15.167:0/2855925370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee8101f40 msgr2=0x7f0ee80741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:42.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0eed93c640 1 --2- 172.21.15.167:0/2855925370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee8101f40 0x7f0ee80741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:42.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0eed93c640 1 -- 172.21.15.167:0/2855925370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ee8101590 msgr2=0x7f0ee8073ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:42.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0eed93c640 1 --2- 172.21.15.167:0/2855925370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ee8101590 0x7f0ee8073ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:42.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0eed93c640 1 -- 172.21.15.167:0/2855925370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee81a5650 con 0x7f0ee81066c0 2024-06-18T13:03:42.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0eec93a640 1 --2- 172.21.15.167:0/2855925370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee8101f40 0x7f0ee80741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:03:42.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0eed93c640 1 --2- 172.21.15.167:0/2855925370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee81066c0 0x7f0ee8074720 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000efc0 tx=0x7f0ee000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:42.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0ede7fc640 1 -- 172.21.15.167:0/2855925370 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0003970 con 0x7f0ee81066c0 2024-06-18T13:03:42.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0ede7fc640 1 -- 172.21.15.167:0/2855925370 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f0ee0004020 con 0x7f0ee81066c0 2024-06-18T13:03:42.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0eef39f640 1 -- 172.21.15.167:0/2855925370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee81a5940 con 0x7f0ee81066c0 2024-06-18T13:03:42.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0ede7fc640 1 -- 172.21.15.167:0/2855925370 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee000b870 con 0x7f0ee81066c0 2024-06-18T13:03:42.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.698+0000 7f0eef39f640 1 -- 172.21.15.167:0/2855925370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee81a5e80 con 0x7f0ee81066c0 2024-06-18T13:03:42.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.700+0000 7f0eef39f640 1 -- 172.21.15.167:0/2855925370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee8101970 con 0x7f0ee81066c0 2024-06-18T13:03:42.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.700+0000 7f0ede7fc640 1 -- 172.21.15.167:0/2855925370 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f0ee001d020 con 0x7f0ee81066c0 2024-06-18T13:03:42.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.700+0000 7f0ede7fc640 1 --2- 172.21.15.167:0/2855925370 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f0ed406e040 0x7f0ed4070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:42.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.700+0000 7f0ede7fc640 1 -- 172.21.15.167:0/2855925370 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7f0ee008e350 con 0x7f0ee81066c0 2024-06-18T13:03:42.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.700+0000 7f0eed13b640 1 --2- 172.21.15.167:0/2855925370 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f0ed406e040 0x7f0ed4070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:42.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.701+0000 7f0eed13b640 1 --2- 172.21.15.167:0/2855925370 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f0ed406e040 0x7f0ed4070500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009830 tx=0x7f0ee40058d0 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:42.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.705+0000 7f0ede7fc640 1 -- 172.21.15.167:0/2855925370 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f0ee005f480 con 0x7f0ee81066c0 2024-06-18T13:03:42.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:42 smithi167 ceph-mon[37701]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:42.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3974978733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-18T13:03:42.841 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:42 smithi167 ceph-mon[29496]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:42.841 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3974978733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-18T13:03:42.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.883+0000 7f0eef39f640 1 -- 172.21.15.167:0/2855925370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0ee81a6110 con 0x7f0ee81066c0 2024-06-18T13:03:42.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.884+0000 7f0ede7fc640 1 -- 172.21.15.167:0/2855925370 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0ee0063cb0 con 0x7f0ee81066c0 2024-06-18T13:03:42.883 INFO:teuthology.orchestra.run.smithi167.stdout:201863462936 2024-06-18T13:03:42.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.886+0000 7f0eef39f640 1 -- 172.21.15.167:0/2855925370 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f0ed406e040 msgr2=0x7f0ed4070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:42.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.886+0000 7f0eef39f640 1 --2- 172.21.15.167:0/2855925370 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f0ed406e040 0x7f0ed4070500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009830 tx=0x7f0ee40058d0 comp rx=0 tx=0).stop 2024-06-18T13:03:42.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.886+0000 7f0eef39f640 1 -- 172.21.15.167:0/2855925370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee81066c0 msgr2=0x7f0ee8074720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:42.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.886+0000 7f0eef39f640 1 --2- 172.21.15.167:0/2855925370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee81066c0 0x7f0ee8074720 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000efc0 tx=0x7f0ee000c800 comp rx=0 tx=0).stop 2024-06-18T13:03:42.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.886+0000 7f0eef39f640 1 -- 172.21.15.167:0/2855925370 shutdown_connections 2024-06-18T13:03:42.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.886+0000 7f0eef39f640 1 --2- 172.21.15.167:0/2855925370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee81066c0 0x7f0ee8074720 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:42.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.886+0000 7f0eef39f640 1 --2- 172.21.15.167:0/2855925370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee8101f40 0x7f0ee80741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:42.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.886+0000 7f0eef39f640 1 --2- 172.21.15.167:0/2855925370 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f0ed406e040 0x7f0ed4070500 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:42.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.886+0000 7f0eef39f640 1 --2- 172.21.15.167:0/2855925370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ee8101590 0x7f0ee8073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:42.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.886+0000 7f0eef39f640 1 -- 172.21.15.167:0/2855925370 >> 172.21.15.167:0/2855925370 conn(0x7f0ee80fd070 msgr2=0x7f0ee80fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:42.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.886+0000 7f0eef39f640 1 -- 172.21.15.167:0/2855925370 shutdown_connections 2024-06-18T13:03:42.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:42.886+0000 7f0eef39f640 1 -- 172.21.15.167:0/2855925370 wait complete. 2024-06-18T13:03:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:42 smithi179 ceph-mon[33312]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3974978733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-18T13:03:43.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2855925370' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-18T13:03:43.665 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2855925370' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-18T13:03:43.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2855925370' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-18T13:03:43.949 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462933 got 201863462936 for osd.6 2024-06-18T13:03:43.949 DEBUG:teuthology.parallel:result is None 2024-06-18T13:03:44.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:44 smithi179 ceph-mon[33312]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:44.903 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:44 smithi167 ceph-mon[37701]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:44.903 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:44 smithi167 ceph-mon[29496]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:44.933 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168591 got 240518168596 for osd.7 2024-06-18T13:03:44.933 DEBUG:teuthology.parallel:result is None 2024-06-18T13:03:45.214 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:03:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:03:45.245 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574891 got 55834574895 for osd.1 2024-06-18T13:03:45.245 DEBUG:teuthology.parallel:result is None 2024-06-18T13:03:45.788 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:45.788 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:46.767 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:46 smithi167 ceph-mon[37701]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:46.768 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:46 smithi167 ceph-mon[29496]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:46.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:46 smithi179 ceph-mon[33312]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:47.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.917+0000 7fe980691640 1 -- 172.21.15.167:0/1607244421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9781055c0 msgr2=0x7fe97810d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:47.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.917+0000 7fe980691640 1 --2- 172.21.15.167:0/1607244421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9781055c0 0x7fe97810d140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe970009ca0 tx=0x7fe97002a8c0 comp rx=0 tx=0).stop 2024-06-18T13:03:47.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.917+0000 7fe980691640 1 -- 172.21.15.167:0/1607244421 shutdown_connections 2024-06-18T13:03:47.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.917+0000 7fe980691640 1 --2- 172.21.15.167:0/1607244421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9781055c0 0x7fe97810d140 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:47.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.917+0000 7fe980691640 1 --2- 172.21.15.167:0/1607244421 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe978100e40 0x7fe978104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:47.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.917+0000 7fe980691640 1 --2- 172.21.15.167:0/1607244421 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe978100490 0x7fe978100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:47.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.917+0000 7fe980691640 1 -- 172.21.15.167:0/1607244421 >> 172.21.15.167:0/1607244421 conn(0x7fe9780fbf50 msgr2=0x7fe9780fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:47.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.917+0000 7fe980691640 1 -- 172.21.15.167:0/1607244421 shutdown_connections 2024-06-18T13:03:47.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.917+0000 7fe980691640 1 -- 172.21.15.167:0/1607244421 wait complete. 2024-06-18T13:03:47.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.918+0000 7fe980691640 1 Processor -- start 2024-06-18T13:03:47.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.918+0000 7fe980691640 1 -- start start 2024-06-18T13:03:47.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.918+0000 7fe980691640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe978100490 0x7fe978197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:47.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.918+0000 7fe980691640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe978100e40 0x7fe978197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:47.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.918+0000 7fe980691640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9781055c0 0x7fe97819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:47.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.918+0000 7fe980691640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe978110bf0 con 0x7fe978100490 2024-06-18T13:03:47.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.918+0000 7fe980691640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe978110a70 con 0x7fe9781055c0 2024-06-18T13:03:47.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.918+0000 7fe980691640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe978110d70 con 0x7fe978100e40 2024-06-18T13:03:47.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe97dc2c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe978100e40 0x7fe978197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:47.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe97dc2c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe978100e40 0x7fe978197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:60856/0 (socket says 172.21.15.167:60856) 2024-06-18T13:03:47.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe97dc2c640 1 -- 172.21.15.167:0/663698432 learned_addr learned my addr 172.21.15.167:0/663698432 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:47.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe97e42d640 1 --2- 172.21.15.167:0/663698432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe978100490 0x7fe978197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:47.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe97ec2e640 1 --2- 172.21.15.167:0/663698432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9781055c0 0x7fe97819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:47.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe97dc2c640 1 -- 172.21.15.167:0/663698432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9781055c0 msgr2=0x7fe97819b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:47.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe97dc2c640 1 --2- 172.21.15.167:0/663698432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9781055c0 0x7fe97819b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:47.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe97dc2c640 1 -- 172.21.15.167:0/663698432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe978100490 msgr2=0x7fe978197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:47.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe97dc2c640 1 --2- 172.21.15.167:0/663698432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe978100490 0x7fe978197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:47.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe97dc2c640 1 -- 172.21.15.167:0/663698432 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97819c0c0 con 0x7fe978100e40 2024-06-18T13:03:47.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe97dc2c640 1 --2- 172.21.15.167:0/663698432 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe978100e40 0x7fe978197650 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe96800e9f0 tx=0x7fe96800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:47.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe96f7fe640 1 -- 172.21.15.167:0/663698432 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96800cd80 con 0x7fe978100e40 2024-06-18T13:03:47.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe96f7fe640 1 -- 172.21.15.167:0/663698432 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fe9680054a0 con 0x7fe978100e40 2024-06-18T13:03:47.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe980691640 1 -- 172.21.15.167:0/663698432 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97819c3b0 con 0x7fe978100e40 2024-06-18T13:03:47.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe96f7fe640 1 -- 172.21.15.167:0/663698432 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968010430 con 0x7fe978100e40 2024-06-18T13:03:47.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.919+0000 7fe980691640 1 -- 172.21.15.167:0/663698432 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9781a3c90 con 0x7fe978100e40 2024-06-18T13:03:47.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.921+0000 7fe980691640 1 -- 172.21.15.167:0/663698432 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe978100870 con 0x7fe978100e40 2024-06-18T13:03:47.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.921+0000 7fe96f7fe640 1 -- 172.21.15.167:0/663698432 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7fe968004fe0 con 0x7fe978100e40 2024-06-18T13:03:47.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.921+0000 7fe96f7fe640 1 --2- 172.21.15.167:0/663698432 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fe96406e040 0x7fe964070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:47.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.921+0000 7fe96f7fe640 1 -- 172.21.15.167:0/663698432 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7fe96808e8d0 con 0x7fe978100e40 2024-06-18T13:03:47.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.922+0000 7fe97e42d640 1 --2- 172.21.15.167:0/663698432 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fe96406e040 0x7fe964070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:47.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.922+0000 7fe97e42d640 1 --2- 172.21.15.167:0/663698432 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fe96406e040 0x7fe964070500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe97400bbe0 tx=0x7fe97400b390 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:47.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:47.926+0000 7fe96f7fe640 1 -- 172.21.15.167:0/663698432 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fe96805fac0 con 0x7fe978100e40 2024-06-18T13:03:48.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:48.109+0000 7fe980691640 1 -- 172.21.15.167:0/663698432 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe978108800 con 0x7fe978100e40 2024-06-18T13:03:48.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:48.109+0000 7fe96f7fe640 1 -- 172.21.15.167:0/663698432 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe9680642f0 con 0x7fe978100e40 2024-06-18T13:03:48.109 INFO:teuthology.orchestra.run.smithi167.stdout:111669149735 2024-06-18T13:03:48.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:48.111+0000 7fe980691640 1 -- 172.21.15.167:0/663698432 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fe96406e040 msgr2=0x7fe964070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:48.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:48.111+0000 7fe980691640 1 --2- 172.21.15.167:0/663698432 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fe96406e040 0x7fe964070500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe97400bbe0 tx=0x7fe97400b390 comp rx=0 tx=0).stop 2024-06-18T13:03:48.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:48.111+0000 7fe980691640 1 -- 172.21.15.167:0/663698432 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe978100e40 msgr2=0x7fe978197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:48.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:48.111+0000 7fe980691640 1 --2- 172.21.15.167:0/663698432 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe978100e40 0x7fe978197650 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe96800e9f0 tx=0x7fe96800edf0 comp rx=0 tx=0).stop 2024-06-18T13:03:48.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:48.111+0000 7fe980691640 1 -- 172.21.15.167:0/663698432 shutdown_connections 2024-06-18T13:03:48.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:48.111+0000 7fe980691640 1 --2- 172.21.15.167:0/663698432 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fe96406e040 0x7fe964070500 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:48.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:48.111+0000 7fe980691640 1 --2- 172.21.15.167:0/663698432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9781055c0 0x7fe97819b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:48.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:48.111+0000 7fe980691640 1 --2- 172.21.15.167:0/663698432 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe978100e40 0x7fe978197650 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:48.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:48.111+0000 7fe980691640 1 --2- 172.21.15.167:0/663698432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe978100490 0x7fe978197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:48.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:48.111+0000 7fe980691640 1 -- 172.21.15.167:0/663698432 >> 172.21.15.167:0/663698432 conn(0x7fe9780fbf50 msgr2=0x7fe9780fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:48.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:48.111+0000 7fe980691640 1 -- 172.21.15.167:0/663698432 shutdown_connections 2024-06-18T13:03:48.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:48.111+0000 7fe980691640 1 -- 172.21.15.167:0/663698432 wait complete. 2024-06-18T13:03:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:48 smithi167 ceph-mon[29496]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/663698432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-18T13:03:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:48 smithi167 ceph-mon[37701]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:48.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/663698432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-18T13:03:48.728 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:48 smithi167 podman[71392]: 2024-06-18 13:03:48.616250864 +0000 UTC m=+0.637950181 container create 44dbeec8b46cd8bcd5b8045d108218f2949b010b6c47928873954e444dc41fa2 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-18T13:03:48.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:03:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:03:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:48 smithi179 ceph-mon[33312]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/663698432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-18T13:03:49.051 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:48 smithi167 podman[71392]: 2024-06-18 13:03:48.902833171 +0000 UTC m=+0.924532495 container init 44dbeec8b46cd8bcd5b8045d108218f2949b010b6c47928873954e444dc41fa2 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-18T13:03:49.051 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:48 smithi167 podman[71392]: 2024-06-18 13:03:48.905800897 +0000 UTC m=+0.927500215 container start 44dbeec8b46cd8bcd5b8045d108218f2949b010b6c47928873954e444dc41fa2 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-18T13:03:49.051 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:48 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[71439]: ts=2024-06-18T13:03:48.918Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-06-18T13:03:49.052 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:48 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[71439]: ts=2024-06-18T13:03:48.918Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-06-18T13:03:49.052 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:48 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[71439]: ts=2024-06-18T13:03:48.918Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.167 port=9094 2024-06-18T13:03:49.052 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:48 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[71439]: ts=2024-06-18T13:03:48.920Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-06-18T13:03:49.052 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:48 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[71439]: ts=2024-06-18T13:03:48.943Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-18T13:03:49.052 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:48 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[71439]: ts=2024-06-18T13:03:48.943Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-18T13:03:49.052 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:48 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[71439]: ts=2024-06-18T13:03:48.946Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-06-18T13:03:49.052 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:48 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[71439]: ts=2024-06-18T13:03:48.946Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-06-18T13:03:49.052 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:49 smithi167 bash[71392]: 44dbeec8b46cd8bcd5b8045d108218f2949b010b6c47928873954e444dc41fa2 2024-06-18T13:03:49.052 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:49 smithi167 systemd[1]: Started Ceph alertmanager.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:03:49.380 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:49 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:03:49] ENGINE Bus STOPPING 2024-06-18T13:03:49.631 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:49 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:03:49] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-18T13:03:49.631 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:49 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:03:49] ENGINE Bus STOPPED 2024-06-18T13:03:49.631 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:49 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:03:49] ENGINE Bus STARTING 2024-06-18T13:03:49.663 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149730 got 111669149735 for osd.3 2024-06-18T13:03:49.663 DEBUG:teuthology.parallel:result is None 2024-06-18T13:03:49.933 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:49 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:03:49] ENGINE Serving on http://:::9283 2024-06-18T13:03:49.933 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:49 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:03:49] ENGINE Bus STARTED 2024-06-18T13:03:50.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.149+0000 7f29addd0640 1 -- 172.21.15.167:0/1147515797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29a810b930 msgr2=0x7f29a810ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:50.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.149+0000 7f29addd0640 1 --2- 172.21.15.167:0/1147515797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29a810b930 0x7f29a810ddb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009860 tx=0x7f29a002a3d0 comp rx=0 tx=0).stop 2024-06-18T13:03:50.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.150+0000 7f29addd0640 1 -- 172.21.15.167:0/1147515797 shutdown_connections 2024-06-18T13:03:50.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.150+0000 7f29addd0640 1 --2- 172.21.15.167:0/1147515797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29a810b930 0x7f29a810ddb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:50.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.150+0000 7f29addd0640 1 --2- 172.21.15.167:0/1147515797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a8102e70 0x7f29a810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:50.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.150+0000 7f29addd0640 1 --2- 172.21.15.167:0/1147515797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a8102550 0x7f29a8102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:50.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.150+0000 7f29addd0640 1 -- 172.21.15.167:0/1147515797 >> 172.21.15.167:0/1147515797 conn(0x7f29a80fbf50 msgr2=0x7f29a80fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:50.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.150+0000 7f29addd0640 1 -- 172.21.15.167:0/1147515797 shutdown_connections 2024-06-18T13:03:50.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.150+0000 7f29addd0640 1 -- 172.21.15.167:0/1147515797 wait complete. 2024-06-18T13:03:50.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.150+0000 7f29addd0640 1 Processor -- start 2024-06-18T13:03:50.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.150+0000 7f29addd0640 1 -- start start 2024-06-18T13:03:50.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29addd0640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29a8102550 0x7f29a81001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:50.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29addd0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a8102e70 0x7f29a81006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:50.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29addd0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a810b930 0x7f29a8100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:50.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29addd0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8110b40 con 0x7f29a8102e70 2024-06-18T13:03:50.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29addd0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81109c0 con 0x7f29a810b930 2024-06-18T13:03:50.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29addd0640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f29a8110cc0 con 0x7f29a8102550 2024-06-18T13:03:50.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29a8102550 0x7f29a81001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:50.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29a8102550 0x7f29a81001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:60878/0 (socket says 172.21.15.167:60878) 2024-06-18T13:03:50.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29a77fe640 1 -- 172.21.15.167:0/2061631813 learned_addr learned my addr 172.21.15.167:0/2061631813 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:50.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29a6ffd640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a8102e70 0x7f29a81006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:50.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29a7fff640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a810b930 0x7f29a8100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:50.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29a6ffd640 1 -- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29a8102550 msgr2=0x7f29a81001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:50.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29a6ffd640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29a8102550 0x7f29a81001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:50.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29a6ffd640 1 -- 172.21.15.167:0/2061631813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a810b930 msgr2=0x7f29a8100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:50.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29a6ffd640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a810b930 0x7f29a8100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:50.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29a6ffd640 1 -- 172.21.15.167:0/2061631813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a81a5590 con 0x7f29a8102e70 2024-06-18T13:03:50.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29a77fe640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29a8102550 0x7f29a81001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:03:50.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29a6ffd640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a8102e70 0x7f29a81006f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f299c005fd0 tx=0x7f299c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:50.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29a4ff9640 1 -- 172.21.15.167:0/2061631813 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c00cd00 con 0x7f29a8102e70 2024-06-18T13:03:50.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29addd0640 1 -- 172.21.15.167:0/2061631813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a81a5880 con 0x7f29a8102e70 2024-06-18T13:03:50.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.151+0000 7f29a4ff9640 1 -- 172.21.15.167:0/2061631813 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f299c0035f0 con 0x7f29a8102e70 2024-06-18T13:03:50.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.152+0000 7f29a4ff9640 1 -- 172.21.15.167:0/2061631813 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c0044a0 con 0x7f29a8102e70 2024-06-18T13:03:50.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.152+0000 7f29addd0640 1 -- 172.21.15.167:0/2061631813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a81a5e10 con 0x7f29a8102e70 2024-06-18T13:03:50.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.152+0000 7f29addd0640 1 -- 172.21.15.167:0/2061631813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a8102930 con 0x7f29a8102e70 2024-06-18T13:03:50.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.153+0000 7f29a4ff9640 1 -- 172.21.15.167:0/2061631813 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f299c004000 con 0x7f29a8102e70 2024-06-18T13:03:50.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.153+0000 7f29a4ff9640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f299006e040 0x7f2990070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:50.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.154+0000 7f29a4ff9640 1 -- 172.21.15.167:0/2061631813 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7f299c08e740 con 0x7f29a8102e70 2024-06-18T13:03:50.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.154+0000 7f29a77fe640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f299006e040 0x7f2990070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:50.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.154+0000 7f29a77fe640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f299006e040 0x7f2990070500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f299800f710 tx=0x7f2998007670 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:50.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.158+0000 7f29a4ff9640 1 -- 172.21.15.167:0/2061631813 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f299c05f7f0 con 0x7f29a8102e70 2024-06-18T13:03:50.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:03:50.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[29496]: Regenerating cephadm self-signed grafana TLS certificates 2024-06-18T13:03:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-18T13:03:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[29496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-18T13:03:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[29496]: Deploying daemon grafana.a on smithi179 2024-06-18T13:03:50.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:03:50.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[37701]: Regenerating cephadm self-signed grafana TLS certificates 2024-06-18T13:03:50.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-18T13:03:50.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[37701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-18T13:03:50.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-mon[37701]: Deploying daemon grafana.a on smithi179 2024-06-18T13:03:50.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.340+0000 7f29addd0640 1 -- 172.21.15.167:0/2061631813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f29a81a60a0 con 0x7f29a8102e70 2024-06-18T13:03:50.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.340+0000 7f29a4ff9640 1 -- 172.21.15.167:0/2061631813 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f299c064020 con 0x7f29a8102e70 2024-06-18T13:03:50.340 INFO:teuthology.orchestra.run.smithi167.stdout:77309411372 2024-06-18T13:03:50.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.342+0000 7f29addd0640 1 -- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f299006e040 msgr2=0x7f2990070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:50.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.342+0000 7f29addd0640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f299006e040 0x7f2990070500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f299800f710 tx=0x7f2998007670 comp rx=0 tx=0).stop 2024-06-18T13:03:50.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.342+0000 7f29addd0640 1 -- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a8102e70 msgr2=0x7f29a81006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.342+0000 7f29addd0640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a8102e70 0x7f29a81006f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f299c005fd0 tx=0x7f299c00ee50 comp rx=0 tx=0).stop 2024-06-18T13:03:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.342+0000 7f29addd0640 1 -- 172.21.15.167:0/2061631813 shutdown_connections 2024-06-18T13:03:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.342+0000 7f29addd0640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f299006e040 0x7f2990070500 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.342+0000 7f29addd0640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a810b930 0x7f29a8100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.342+0000 7f29addd0640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a8102e70 0x7f29a81006f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.342+0000 7f29addd0640 1 --2- 172.21.15.167:0/2061631813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29a8102550 0x7f29a81001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.342+0000 7f29addd0640 1 -- 172.21.15.167:0/2061631813 >> 172.21.15.167:0/2061631813 conn(0x7f29a80fbf50 msgr2=0x7f29a810cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.342+0000 7f29addd0640 1 -- 172.21.15.167:0/2061631813 shutdown_connections 2024-06-18T13:03:50.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:50.342+0000 7f29addd0640 1 -- 172.21.15.167:0/2061631813 wait complete. 2024-06-18T13:03:50.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:03:50.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:50 smithi179 ceph-mon[33312]: Regenerating cephadm self-signed grafana TLS certificates 2024-06-18T13:03:50.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-18T13:03:50.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:50 smithi179 ceph-mon[33312]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-18T13:03:50.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:50.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:50 smithi179 ceph-mon[33312]: Deploying daemon grafana.a on smithi179 2024-06-18T13:03:50.949 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411366 got 77309411372 for osd.2 2024-06-18T13:03:50.949 DEBUG:teuthology.parallel:result is None 2024-06-18T13:03:50.949 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-18T13:03:50.949 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph pg dump --format=json 2024-06-18T13:03:51.189 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:51.201 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:50 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[71439]: ts=2024-06-18T13:03:50.920Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000259821s 2024-06-18T13:03:51.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:51 smithi167 ceph-mon[37701]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:51.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:51 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2061631813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-18T13:03:51.456 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:51 smithi167 ceph-mon[29496]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:51.457 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:51 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2061631813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-18T13:03:51.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:51 smithi179 ceph-mon[33312]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:51.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:51 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2061631813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-18T13:03:52.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.923+0000 7f37c2abf640 1 -- 172.21.15.167:0/4275258185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37bc100e40 msgr2=0x7f37bc104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:52.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.923+0000 7f37c2abf640 1 --2- 172.21.15.167:0/4275258185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37bc100e40 0x7f37bc104e90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009ca0 tx=0x7f37ac02a8c0 comp rx=0 tx=0).stop 2024-06-18T13:03:52.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.923+0000 7f37c2abf640 1 -- 172.21.15.167:0/4275258185 shutdown_connections 2024-06-18T13:03:52.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.923+0000 7f37c2abf640 1 --2- 172.21.15.167:0/4275258185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37bc1055c0 0x7f37bc10d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:52.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.923+0000 7f37c2abf640 1 --2- 172.21.15.167:0/4275258185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37bc100e40 0x7f37bc104e90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:52.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.923+0000 7f37c2abf640 1 --2- 172.21.15.167:0/4275258185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37bc100490 0x7f37bc100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.923+0000 7f37c2abf640 1 -- 172.21.15.167:0/4275258185 >> 172.21.15.167:0/4275258185 conn(0x7f37bc0fbf50 msgr2=0x7f37bc0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.923+0000 7f37c2abf640 1 -- 172.21.15.167:0/4275258185 shutdown_connections 2024-06-18T13:03:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.923+0000 7f37c2abf640 1 -- 172.21.15.167:0/4275258185 wait complete. 2024-06-18T13:03:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c2abf640 1 Processor -- start 2024-06-18T13:03:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c2abf640 1 -- start start 2024-06-18T13:03:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c2abf640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37bc100490 0x7f37bc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c2abf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37bc100e40 0x7f37bc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c2abf640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37bc1055c0 0x7f37bc1a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c2abf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc10fa60 con 0x7f37bc100e40 2024-06-18T13:03:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c2abf640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc10f8e0 con 0x7f37bc1055c0 2024-06-18T13:03:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c2abf640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f37bc10fbe0 con 0x7f37bc100490 2024-06-18T13:03:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c085b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37bc100490 0x7f37bc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c085b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37bc100490 0x7f37bc197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:60906/0 (socket says 172.21.15.167:60906) 2024-06-18T13:03:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c085b640 1 -- 172.21.15.167:0/2789657270 learned_addr learned my addr 172.21.15.167:0/2789657270 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37bbfff640 1 --2- 172.21.15.167:0/2789657270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37bc100e40 0x7f37bc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c105c640 1 --2- 172.21.15.167:0/2789657270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37bc1055c0 0x7f37bc1a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c085b640 1 -- 172.21.15.167:0/2789657270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37bc1055c0 msgr2=0x7f37bc1a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c085b640 1 --2- 172.21.15.167:0/2789657270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37bc1055c0 0x7f37bc1a3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c085b640 1 -- 172.21.15.167:0/2789657270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37bc100e40 msgr2=0x7f37bc1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c085b640 1 --2- 172.21.15.167:0/2789657270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37bc100e40 0x7f37bc1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c085b640 1 -- 172.21.15.167:0/2789657270 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc1a3560 con 0x7f37bc100490 2024-06-18T13:03:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.924+0000 7f37c085b640 1 --2- 172.21.15.167:0/2789657270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37bc100490 0x7f37bc197fa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f37b400d950 tx=0x7f37b400dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.925+0000 7f37b9ffb640 1 -- 172.21.15.167:0/2789657270 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4017070 con 0x7f37bc100490 2024-06-18T13:03:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.925+0000 7f37c2abf640 1 -- 172.21.15.167:0/2789657270 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37bc1a3730 con 0x7f37bc100490 2024-06-18T13:03:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.925+0000 7f37b9ffb640 1 -- 172.21.15.167:0/2789657270 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f37b400de80 con 0x7f37bc100490 2024-06-18T13:03:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.925+0000 7f37b9ffb640 1 -- 172.21.15.167:0/2789657270 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b400f620 con 0x7f37bc100490 2024-06-18T13:03:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.925+0000 7f37c2abf640 1 -- 172.21.15.167:0/2789657270 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f37bc1a3c70 con 0x7f37bc100490 2024-06-18T13:03:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.926+0000 7f37c2abf640 1 -- 172.21.15.167:0/2789657270 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37bc100870 con 0x7f37bc100490 2024-06-18T13:03:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.926+0000 7f37b9ffb640 1 -- 172.21.15.167:0/2789657270 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f37b4010460 con 0x7f37bc100490 2024-06-18T13:03:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.927+0000 7f37b9ffb640 1 --2- 172.21.15.167:0/2789657270 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f37a806e040 0x7f37a8070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.927+0000 7f37bbfff640 1 --2- 172.21.15.167:0/2789657270 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f37a806e040 0x7f37a8070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.927+0000 7f37b9ffb640 1 -- 172.21.15.167:0/2789657270 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7f37b408e1b0 con 0x7f37bc100490 2024-06-18T13:03:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.927+0000 7f37bbfff640 1 --2- 172.21.15.167:0/2789657270 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f37a806e040 0x7f37a8070500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f37bc06aa90 tx=0x7f37ac009360 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:52.931+0000 7f37b9ffb640 1 -- 172.21.15.167:0/2789657270 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f37b4010e20 con 0x7f37bc100490 2024-06-18T13:03:53.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:53.105+0000 7f37c2abf640 1 -- 172.21.15.167:0/2789657270 --> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37bc0fccb0 con 0x7f37a806e040 2024-06-18T13:03:53.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:53.114+0000 7f37b9ffb640 1 -- 172.21.15.167:0/2789657270 <== mgr.24461 v2:172.21.15.167:6800/2104657861 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346474 (secure 0 0 0) 0x7f37bc0fccb0 con 0x7f37a806e040 2024-06-18T13:03:53.117 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T13:03:53.118 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-06-18T13:03:53.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:53.117+0000 7f37c2abf640 1 -- 172.21.15.167:0/2789657270 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f37a806e040 msgr2=0x7f37a8070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:53.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:53.117+0000 7f37c2abf640 1 --2- 172.21.15.167:0/2789657270 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f37a806e040 0x7f37a8070500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f37bc06aa90 tx=0x7f37ac009360 comp rx=0 tx=0).stop 2024-06-18T13:03:53.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:53.117+0000 7f37c2abf640 1 -- 172.21.15.167:0/2789657270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37bc100490 msgr2=0x7f37bc197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:53.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:53.117+0000 7f37c2abf640 1 --2- 172.21.15.167:0/2789657270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37bc100490 0x7f37bc197fa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f37b400d950 tx=0x7f37b400dd50 comp rx=0 tx=0).stop 2024-06-18T13:03:53.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:53.117+0000 7f37c2abf640 1 -- 172.21.15.167:0/2789657270 shutdown_connections 2024-06-18T13:03:53.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:53.117+0000 7f37c2abf640 1 --2- 172.21.15.167:0/2789657270 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f37a806e040 0x7f37a8070500 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:53.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:53.117+0000 7f37c2abf640 1 --2- 172.21.15.167:0/2789657270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37bc1055c0 0x7f37bc1a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:53.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:53.117+0000 7f37c2abf640 1 --2- 172.21.15.167:0/2789657270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37bc100e40 0x7f37bc1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:53.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:53.117+0000 7f37c2abf640 1 --2- 172.21.15.167:0/2789657270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37bc100490 0x7f37bc197fa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:53.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:53.117+0000 7f37c2abf640 1 -- 172.21.15.167:0/2789657270 >> 172.21.15.167:0/2789657270 conn(0x7f37bc0fbf50 msgr2=0x7f37bc0fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:53.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:53.117+0000 7f37c2abf640 1 -- 172.21.15.167:0/2789657270 shutdown_connections 2024-06-18T13:03:53.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:53.117+0000 7f37c2abf640 1 -- 172.21.15.167:0/2789657270 wait complete. 2024-06-18T13:03:53.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:53 smithi167 ceph-mon[37701]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:53.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:53 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:53.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:53 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:03:53.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:53 smithi167 ceph-mon[29496]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:53.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:53 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:53.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:53 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:03:53.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:53 smithi179 ceph-mon[33312]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:53.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:53 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:03:53.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:53 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:03:53.798 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-06-18T13:03:51.439300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1005,"num_read_kb":851,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":249076,"kb_used_data":7964,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":749712140,"statfs":{"total":767960285184,"available":767705231360,"internally_reserved":0,"allocated":8155136,"data_stored":4378985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003135"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419508+0000","last_change":"2024-06-18T13:02:36.472243+0000","last_active":"2024-06-18T13:03:07.419508+0000","last_peered":"2024-06-18T13:03:07.419508+0000","last_clean":"2024-06-18T13:03:07.419508+0000","last_became_active":"2024-06-18T13:02:36.471928+0000","last_became_peered":"2024-06-18T13:02:36.471928+0000","last_unstale":"2024-06-18T13:03:07.419508+0000","last_undegraded":"2024-06-18T13:03:07.419508+0000","last_fullsized":"2024-06-18T13:03:07.419508+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:12:16.543101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444127+0000","last_change":"2024-06-18T13:02:30.418875+0000","last_active":"2024-06-18T13:03:07.444127+0000","last_peered":"2024-06-18T13:03:07.444127+0000","last_clean":"2024-06-18T13:03:07.444127+0000","last_became_active":"2024-06-18T13:02:30.418554+0000","last_became_peered":"2024-06-18T13:02:30.418554+0000","last_unstale":"2024-06-18T13:03:07.444127+0000","last_undegraded":"2024-06-18T13:03:07.444127+0000","last_fullsized":"2024-06-18T13:03:07.444127+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:22:32.818017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"69'15","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419757+0000","last_change":"2024-06-18T13:02:32.416501+0000","last_active":"2024-06-18T13:03:07.419757+0000","last_peered":"2024-06-18T13:03:07.419757+0000","last_clean":"2024-06-18T13:03:07.419757+0000","last_became_active":"2024-06-18T13:02:32.416134+0000","last_became_peered":"2024-06-18T13:02:32.416134+0000","last_unstale":"2024-06-18T13:03:07.419757+0000","last_undegraded":"2024-06-18T13:03:07.419757+0000","last_fullsized":"2024-06-18T13:03:07.419757+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:21:05.663556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443946+0000","last_change":"2024-06-18T13:02:34.437890+0000","last_active":"2024-06-18T13:03:07.443946+0000","last_peered":"2024-06-18T13:03:07.443946+0000","last_clean":"2024-06-18T13:03:07.443946+0000","last_became_active":"2024-06-18T13:02:34.437560+0000","last_became_peered":"2024-06-18T13:02:34.437560+0000","last_unstale":"2024-06-18T13:03:07.443946+0000","last_undegraded":"2024-06-18T13:03:07.443946+0000","last_fullsized":"2024-06-18T13:03:07.443946+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:32:57.541440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420865+0000","last_change":"2024-06-18T13:02:30.418269+0000","last_active":"2024-06-18T13:03:07.420865+0000","last_peered":"2024-06-18T13:03:07.420865+0000","last_clean":"2024-06-18T13:03:07.420865+0000","last_became_active":"2024-06-18T13:02:30.417989+0000","last_became_peered":"2024-06-18T13:02:30.417989+0000","last_unstale":"2024-06-18T13:03:07.420865+0000","last_undegraded":"2024-06-18T13:03:07.420865+0000","last_fullsized":"2024-06-18T13:03:07.420865+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:31:22.346990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"69'9","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443971+0000","last_change":"2024-06-18T13:02:32.414812+0000","last_active":"2024-06-18T13:03:07.443971+0000","last_peered":"2024-06-18T13:03:07.443971+0000","last_clean":"2024-06-18T13:03:07.443971+0000","last_became_active":"2024-06-18T13:02:32.414482+0000","last_became_peered":"2024-06-18T13:02:32.414482+0000","last_unstale":"2024-06-18T13:03:07.443971+0000","last_undegraded":"2024-06-18T13:03:07.443971+0000","last_fullsized":"2024-06-18T13:03:07.443971+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:40:53.728503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420781+0000","last_change":"2024-06-18T13:02:34.434587+0000","last_active":"2024-06-18T13:03:07.420781+0000","last_peered":"2024-06-18T13:03:07.420781+0000","last_clean":"2024-06-18T13:03:07.420781+0000","last_became_active":"2024-06-18T13:02:34.434292+0000","last_became_peered":"2024-06-18T13:02:34.434292+0000","last_unstale":"2024-06-18T13:03:07.420781+0000","last_undegraded":"2024-06-18T13:03:07.420781+0000","last_fullsized":"2024-06-18T13:03:07.420781+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:35:57.259405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444334+0000","last_change":"2024-06-18T13:02:36.446931+0000","last_active":"2024-06-18T13:03:07.444334+0000","last_peered":"2024-06-18T13:03:07.444334+0000","last_clean":"2024-06-18T13:03:07.444334+0000","last_became_active":"2024-06-18T13:02:36.446627+0000","last_became_peered":"2024-06-18T13:02:36.446627+0000","last_unstale":"2024-06-18T13:03:07.444334+0000","last_undegraded":"2024-06-18T13:03:07.444334+0000","last_fullsized":"2024-06-18T13:03:07.444334+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:53:21.698408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.418484+0000","last_change":"2024-06-18T13:02:30.419267+0000","last_active":"2024-06-18T13:03:07.418484+0000","last_peered":"2024-06-18T13:03:07.418484+0000","last_clean":"2024-06-18T13:03:07.418484+0000","last_became_active":"2024-06-18T13:02:30.419007+0000","last_became_peered":"2024-06-18T13:02:30.419007+0000","last_unstale":"2024-06-18T13:03:07.418484+0000","last_undegraded":"2024-06-18T13:03:07.418484+0000","last_fullsized":"2024-06-18T13:03:07.418484+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:28:59.680188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"69'5","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444548+0000","last_change":"2024-06-18T13:02:32.417988+0000","last_active":"2024-06-18T13:03:07.444548+0000","last_peered":"2024-06-18T13:03:07.444548+0000","last_clean":"2024-06-18T13:03:07.444548+0000","last_became_active":"2024-06-18T13:02:32.417687+0000","last_became_peered":"2024-06-18T13:02:32.417687+0000","last_unstale":"2024-06-18T13:03:07.444548+0000","last_undegraded":"2024-06-18T13:03:07.444548+0000","last_fullsized":"2024-06-18T13:03:07.444548+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:39:46.131680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417463+0000","last_change":"2024-06-18T13:02:34.439133+0000","last_active":"2024-06-18T13:03:07.417463+0000","last_peered":"2024-06-18T13:03:07.417463+0000","last_clean":"2024-06-18T13:03:07.417463+0000","last_became_active":"2024-06-18T13:02:34.438258+0000","last_became_peered":"2024-06-18T13:02:34.438258+0000","last_unstale":"2024-06-18T13:03:07.417463+0000","last_undegraded":"2024-06-18T13:03:07.417463+0000","last_fullsized":"2024-06-18T13:03:07.417463+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:42:21.415594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443611+0000","last_change":"2024-06-18T13:02:36.451296+0000","last_active":"2024-06-18T13:03:07.443611+0000","last_peered":"2024-06-18T13:03:07.443611+0000","last_clean":"2024-06-18T13:03:07.443611+0000","last_became_active":"2024-06-18T13:02:36.450940+0000","last_became_peered":"2024-06-18T13:02:36.450940+0000","last_unstale":"2024-06-18T13:03:07.443611+0000","last_undegraded":"2024-06-18T13:03:07.443611+0000","last_fullsized":"2024-06-18T13:03:07.443611+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:27:56.156544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.418897+0000","last_change":"2024-06-18T13:02:30.423190+0000","last_active":"2024-06-18T13:03:07.418897+0000","last_peered":"2024-06-18T13:03:07.418897+0000","last_clean":"2024-06-18T13:03:07.418897+0000","last_became_active":"2024-06-18T13:02:30.422916+0000","last_became_peered":"2024-06-18T13:02:30.422916+0000","last_unstale":"2024-06-18T13:03:07.418897+0000","last_undegraded":"2024-06-18T13:03:07.418897+0000","last_fullsized":"2024-06-18T13:03:07.418897+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:00:27.122463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"69'9","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444493+0000","last_change":"2024-06-18T13:02:32.417902+0000","last_active":"2024-06-18T13:03:07.444493+0000","last_peered":"2024-06-18T13:03:07.444493+0000","last_clean":"2024-06-18T13:03:07.444493+0000","last_became_active":"2024-06-18T13:02:32.417504+0000","last_became_peered":"2024-06-18T13:02:32.417504+0000","last_unstale":"2024-06-18T13:03:07.444493+0000","last_undegraded":"2024-06-18T13:03:07.444493+0000","last_fullsized":"2024-06-18T13:03:07.444493+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:00:03.673111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444365+0000","last_change":"2024-06-18T13:02:34.441891+0000","last_active":"2024-06-18T13:03:07.444365+0000","last_peered":"2024-06-18T13:03:07.444365+0000","last_clean":"2024-06-18T13:03:07.444365+0000","last_became_active":"2024-06-18T13:02:34.441474+0000","last_became_peered":"2024-06-18T13:02:34.441474+0000","last_unstale":"2024-06-18T13:03:07.444365+0000","last_undegraded":"2024-06-18T13:03:07.444365+0000","last_fullsized":"2024-06-18T13:03:07.444365+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:56:11.258359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443192+0000","last_change":"2024-06-18T13:02:36.452515+0000","last_active":"2024-06-18T13:03:07.443192+0000","last_peered":"2024-06-18T13:03:07.443192+0000","last_clean":"2024-06-18T13:03:07.443192+0000","last_became_active":"2024-06-18T13:02:36.452215+0000","last_became_peered":"2024-06-18T13:02:36.452215+0000","last_unstale":"2024-06-18T13:03:07.443192+0000","last_undegraded":"2024-06-18T13:03:07.443192+0000","last_fullsized":"2024-06-18T13:03:07.443192+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:36:31.678457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"69'17","reported_seq":53,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445059+0000","last_change":"2024-06-18T13:02:32.415203+0000","last_active":"2024-06-18T13:03:07.445059+0000","last_peered":"2024-06-18T13:03:07.445059+0000","last_clean":"2024-06-18T13:03:07.445059+0000","last_became_active":"2024-06-18T13:02:32.414900+0000","last_became_peered":"2024-06-18T13:02:32.414900+0000","last_unstale":"2024-06-18T13:03:07.445059+0000","last_undegraded":"2024-06-18T13:03:07.445059+0000","last_fullsized":"2024-06-18T13:03:07.445059+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:18:51.613696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.418154+0000","last_change":"2024-06-18T13:02:30.418466+0000","last_active":"2024-06-18T13:03:07.418154+0000","last_peered":"2024-06-18T13:03:07.418154+0000","last_clean":"2024-06-18T13:03:07.418154+0000","last_became_active":"2024-06-18T13:02:30.418168+0000","last_became_peered":"2024-06-18T13:02:30.418168+0000","last_unstale":"2024-06-18T13:03:07.418154+0000","last_undegraded":"2024-06-18T13:03:07.418154+0000","last_fullsized":"2024-06-18T13:03:07.418154+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:57:56.080381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.421467+0000","last_change":"2024-06-18T13:02:34.429611+0000","last_active":"2024-06-18T13:03:07.421467+0000","last_peered":"2024-06-18T13:03:07.421467+0000","last_clean":"2024-06-18T13:03:07.421467+0000","last_became_active":"2024-06-18T13:02:34.429344+0000","last_became_peered":"2024-06-18T13:02:34.429344+0000","last_unstale":"2024-06-18T13:03:07.421467+0000","last_undegraded":"2024-06-18T13:03:07.421467+0000","last_fullsized":"2024-06-18T13:03:07.421467+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:24:38.969355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443682+0000","last_change":"2024-06-18T13:02:36.444742+0000","last_active":"2024-06-18T13:03:07.443682+0000","last_peered":"2024-06-18T13:03:07.443682+0000","last_clean":"2024-06-18T13:03:07.443682+0000","last_became_active":"2024-06-18T13:02:36.444411+0000","last_became_peered":"2024-06-18T13:02:36.444411+0000","last_unstale":"2024-06-18T13:03:07.443682+0000","last_undegraded":"2024-06-18T13:03:07.443682+0000","last_fullsized":"2024-06-18T13:03:07.443682+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:07:18.359065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"69'10","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444814+0000","last_change":"2024-06-18T13:02:32.418114+0000","last_active":"2024-06-18T13:03:07.444814+0000","last_peered":"2024-06-18T13:03:07.444814+0000","last_clean":"2024-06-18T13:03:07.444814+0000","last_became_active":"2024-06-18T13:02:32.417669+0000","last_became_peered":"2024-06-18T13:02:32.417669+0000","last_unstale":"2024-06-18T13:03:07.444814+0000","last_undegraded":"2024-06-18T13:03:07.444814+0000","last_fullsized":"2024-06-18T13:03:07.444814+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:12:28.539021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.423542+0000","last_change":"2024-06-18T13:02:30.413373+0000","last_active":"2024-06-18T13:03:07.423542+0000","last_peered":"2024-06-18T13:03:07.423542+0000","last_clean":"2024-06-18T13:03:07.423542+0000","last_became_active":"2024-06-18T13:02:30.412688+0000","last_became_peered":"2024-06-18T13:02:30.412688+0000","last_unstale":"2024-06-18T13:03:07.423542+0000","last_undegraded":"2024-06-18T13:03:07.423542+0000","last_fullsized":"2024-06-18T13:03:07.423542+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T22:34:19.272376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"69'8","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:40.474301+0000","last_change":"2024-06-18T13:02:34.436308+0000","last_active":"2024-06-18T13:03:40.474301+0000","last_peered":"2024-06-18T13:03:40.474301+0000","last_clean":"2024-06-18T13:03:40.474301+0000","last_became_active":"2024-06-18T13:02:34.436033+0000","last_became_peered":"2024-06-18T13:02:34.436033+0000","last_unstale":"2024-06-18T13:03:40.474301+0000","last_undegraded":"2024-06-18T13:03:40.474301+0000","last_fullsized":"2024-06-18T13:03:40.474301+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-20T01:02:01.872653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445414+0000","last_change":"2024-06-18T13:02:36.446884+0000","last_active":"2024-06-18T13:03:07.445414+0000","last_peered":"2024-06-18T13:03:07.445414+0000","last_clean":"2024-06-18T13:03:07.445414+0000","last_became_active":"2024-06-18T13:02:36.446567+0000","last_became_peered":"2024-06-18T13:02:36.446567+0000","last_unstale":"2024-06-18T13:03:07.445414+0000","last_undegraded":"2024-06-18T13:03:07.445414+0000","last_fullsized":"2024-06-18T13:03:07.445414+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-20T00:48:04.981730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"69'15","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.423150+0000","last_change":"2024-06-18T13:02:32.421344+0000","last_active":"2024-06-18T13:03:07.423150+0000","last_peered":"2024-06-18T13:03:07.423150+0000","last_clean":"2024-06-18T13:03:07.423150+0000","last_became_active":"2024-06-18T13:02:32.420960+0000","last_became_peered":"2024-06-18T13:02:32.420960+0000","last_unstale":"2024-06-18T13:03:07.423150+0000","last_undegraded":"2024-06-18T13:03:07.423150+0000","last_fullsized":"2024-06-18T13:03:07.423150+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-20T01:00:53.325751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420976+0000","last_change":"2024-06-18T13:02:30.413272+0000","last_active":"2024-06-18T13:03:07.420976+0000","last_peered":"2024-06-18T13:03:07.420976+0000","last_clean":"2024-06-18T13:03:07.420976+0000","last_became_active":"2024-06-18T13:02:30.412656+0000","last_became_peered":"2024-06-18T13:02:30.412656+0000","last_unstale":"2024-06-18T13:03:07.420976+0000","last_undegraded":"2024-06-18T13:03:07.420976+0000","last_fullsized":"2024-06-18T13:03:07.420976+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:46:18.761210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"69'8","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:40.474392+0000","last_change":"2024-06-18T13:02:34.436572+0000","last_active":"2024-06-18T13:03:40.474392+0000","last_peered":"2024-06-18T13:03:40.474392+0000","last_clean":"2024-06-18T13:03:40.474392+0000","last_became_active":"2024-06-18T13:02:34.435825+0000","last_became_peered":"2024-06-18T13:02:34.435825+0000","last_unstale":"2024-06-18T13:03:40.474392+0000","last_undegraded":"2024-06-18T13:03:40.474392+0000","last_fullsized":"2024-06-18T13:03:40.474392+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:57:01.564982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444674+0000","last_change":"2024-06-18T13:02:36.447260+0000","last_active":"2024-06-18T13:03:07.444674+0000","last_peered":"2024-06-18T13:03:07.444674+0000","last_clean":"2024-06-18T13:03:07.444674+0000","last_became_active":"2024-06-18T13:02:36.446977+0000","last_became_peered":"2024-06-18T13:02:36.446977+0000","last_unstale":"2024-06-18T13:03:07.444674+0000","last_undegraded":"2024-06-18T13:03:07.444674+0000","last_fullsized":"2024-06-18T13:03:07.444674+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:04:48.638790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"69'11","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445182+0000","last_change":"2024-06-18T13:02:32.414932+0000","last_active":"2024-06-18T13:03:07.445182+0000","last_peered":"2024-06-18T13:03:07.445182+0000","last_clean":"2024-06-18T13:03:07.445182+0000","last_became_active":"2024-06-18T13:02:32.414418+0000","last_became_peered":"2024-06-18T13:02:32.414418+0000","last_unstale":"2024-06-18T13:03:07.445182+0000","last_undegraded":"2024-06-18T13:03:07.445182+0000","last_fullsized":"2024-06-18T13:03:07.445182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:34:50.286768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417495+0000","last_change":"2024-06-18T13:02:30.420772+0000","last_active":"2024-06-18T13:03:07.417495+0000","last_peered":"2024-06-18T13:03:07.417495+0000","last_clean":"2024-06-18T13:03:07.417495+0000","last_became_active":"2024-06-18T13:02:30.420481+0000","last_became_peered":"2024-06-18T13:02:30.420481+0000","last_unstale":"2024-06-18T13:03:07.417495+0000","last_undegraded":"2024-06-18T13:03:07.417495+0000","last_fullsized":"2024-06-18T13:03:07.417495+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:06:19.830117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445194+0000","last_change":"2024-06-18T13:02:34.436353+0000","last_active":"2024-06-18T13:03:07.445194+0000","last_peered":"2024-06-18T13:03:07.445194+0000","last_clean":"2024-06-18T13:03:07.445194+0000","last_became_active":"2024-06-18T13:02:34.435995+0000","last_became_peered":"2024-06-18T13:02:34.435995+0000","last_unstale":"2024-06-18T13:03:07.445194+0000","last_undegraded":"2024-06-18T13:03:07.445194+0000","last_fullsized":"2024-06-18T13:03:07.445194+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:39:14.468218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420092+0000","last_change":"2024-06-18T13:02:36.470098+0000","last_active":"2024-06-18T13:03:07.420092+0000","last_peered":"2024-06-18T13:03:07.420092+0000","last_clean":"2024-06-18T13:03:07.420092+0000","last_became_active":"2024-06-18T13:02:36.469759+0000","last_became_peered":"2024-06-18T13:02:36.469759+0000","last_unstale":"2024-06-18T13:03:07.420092+0000","last_undegraded":"2024-06-18T13:03:07.420092+0000","last_fullsized":"2024-06-18T13:03:07.420092+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:27:52.518133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"69'14","reported_seq":46,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443713+0000","last_change":"2024-06-18T13:02:32.414312+0000","last_active":"2024-06-18T13:03:07.443713+0000","last_peered":"2024-06-18T13:03:07.443713+0000","last_clean":"2024-06-18T13:03:07.443713+0000","last_became_active":"2024-06-18T13:02:32.414027+0000","last_became_peered":"2024-06-18T13:02:32.414027+0000","last_unstale":"2024-06-18T13:03:07.443713+0000","last_undegraded":"2024-06-18T13:03:07.443713+0000","last_fullsized":"2024-06-18T13:03:07.443713+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:18:09.361974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'1","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444191+0000","last_change":"2024-06-18T13:02:30.415838+0000","last_active":"2024-06-18T13:03:07.444191+0000","last_peered":"2024-06-18T13:03:07.444191+0000","last_clean":"2024-06-18T13:03:07.444191+0000","last_became_active":"2024-06-18T13:02:30.415576+0000","last_became_peered":"2024-06-18T13:02:30.415576+0000","last_unstale":"2024-06-18T13:03:07.444191+0000","last_undegraded":"2024-06-18T13:03:07.444191+0000","last_fullsized":"2024-06-18T13:03:07.444191+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:57:31.939298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"69'8","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:40.474355+0000","last_change":"2024-06-18T13:02:34.440627+0000","last_active":"2024-06-18T13:03:40.474355+0000","last_peered":"2024-06-18T13:03:40.474355+0000","last_clean":"2024-06-18T13:03:40.474355+0000","last_became_active":"2024-06-18T13:02:34.440273+0000","last_became_peered":"2024-06-18T13:02:34.440273+0000","last_unstale":"2024-06-18T13:03:40.474355+0000","last_undegraded":"2024-06-18T13:03:40.474355+0000","last_fullsized":"2024-06-18T13:03:40.474355+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:14:06.011098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417927+0000","last_change":"2024-06-18T13:02:36.469843+0000","last_active":"2024-06-18T13:03:07.417927+0000","last_peered":"2024-06-18T13:03:07.417927+0000","last_clean":"2024-06-18T13:03:07.417927+0000","last_became_active":"2024-06-18T13:02:36.469587+0000","last_became_peered":"2024-06-18T13:02:36.469587+0000","last_unstale":"2024-06-18T13:03:07.417927+0000","last_undegraded":"2024-06-18T13:03:07.417927+0000","last_fullsized":"2024-06-18T13:03:07.417927+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:52:18.746476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"69'18","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419331+0000","last_change":"2024-06-18T13:02:32.534891+0000","last_active":"2024-06-18T13:03:07.419331+0000","last_peered":"2024-06-18T13:03:07.419331+0000","last_clean":"2024-06-18T13:03:07.419331+0000","last_became_active":"2024-06-18T13:02:32.534560+0000","last_became_peered":"2024-06-18T13:02:32.534560+0000","last_unstale":"2024-06-18T13:03:07.419331+0000","last_undegraded":"2024-06-18T13:03:07.419331+0000","last_fullsized":"2024-06-18T13:03:07.419331+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:46:33.607910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"62'1","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.422810+0000","last_change":"2024-06-18T13:02:30.415028+0000","last_active":"2024-06-18T13:03:07.422810+0000","last_peered":"2024-06-18T13:03:07.422810+0000","last_clean":"2024-06-18T13:03:07.422810+0000","last_became_active":"2024-06-18T13:02:30.414736+0000","last_became_peered":"2024-06-18T13:02:30.414736+0000","last_unstale":"2024-06-18T13:03:07.422810+0000","last_undegraded":"2024-06-18T13:03:07.422810+0000","last_fullsized":"2024-06-18T13:03:07.422810+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:50:27.804864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443486+0000","last_change":"2024-06-18T13:02:34.438695+0000","last_active":"2024-06-18T13:03:07.443486+0000","last_peered":"2024-06-18T13:03:07.443486+0000","last_clean":"2024-06-18T13:03:07.443486+0000","last_became_active":"2024-06-18T13:02:34.438414+0000","last_became_peered":"2024-06-18T13:02:34.438414+0000","last_unstale":"2024-06-18T13:03:07.443486+0000","last_undegraded":"2024-06-18T13:03:07.443486+0000","last_fullsized":"2024-06-18T13:03:07.443486+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:54:28.655422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444440+0000","last_change":"2024-06-18T13:02:36.451124+0000","last_active":"2024-06-18T13:03:07.444440+0000","last_peered":"2024-06-18T13:03:07.444440+0000","last_clean":"2024-06-18T13:03:07.444440+0000","last_became_active":"2024-06-18T13:02:36.450740+0000","last_became_peered":"2024-06-18T13:02:36.450740+0000","last_unstale":"2024-06-18T13:03:07.444440+0000","last_undegraded":"2024-06-18T13:03:07.444440+0000","last_fullsized":"2024-06-18T13:03:07.444440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-20T00:53:37.173230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"69'19","reported_seq":61,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443624+0000","last_change":"2024-06-18T13:02:32.533391+0000","last_active":"2024-06-18T13:03:07.443624+0000","last_peered":"2024-06-18T13:03:07.443624+0000","last_clean":"2024-06-18T13:03:07.443624+0000","last_became_active":"2024-06-18T13:02:32.533062+0000","last_became_peered":"2024-06-18T13:02:32.533062+0000","last_unstale":"2024-06-18T13:03:07.443624+0000","last_undegraded":"2024-06-18T13:03:07.443624+0000","last_fullsized":"2024-06-18T13:03:07.443624+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:53:58.154183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.418568+0000","last_change":"2024-06-18T13:02:30.419269+0000","last_active":"2024-06-18T13:03:07.418568+0000","last_peered":"2024-06-18T13:03:07.418568+0000","last_clean":"2024-06-18T13:03:07.418568+0000","last_became_active":"2024-06-18T13:02:30.418630+0000","last_became_peered":"2024-06-18T13:02:30.418630+0000","last_unstale":"2024-06-18T13:03:07.418568+0000","last_undegraded":"2024-06-18T13:03:07.418568+0000","last_fullsized":"2024-06-18T13:03:07.418568+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-20T00:31:55.971561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443243+0000","last_change":"2024-06-18T13:02:34.444164+0000","last_active":"2024-06-18T13:03:07.443243+0000","last_peered":"2024-06-18T13:03:07.443243+0000","last_clean":"2024-06-18T13:03:07.443243+0000","last_became_active":"2024-06-18T13:02:34.443851+0000","last_became_peered":"2024-06-18T13:02:34.443851+0000","last_unstale":"2024-06-18T13:03:07.443243+0000","last_undegraded":"2024-06-18T13:03:07.443243+0000","last_fullsized":"2024-06-18T13:03:07.443243+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:25:41.611721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420699+0000","last_change":"2024-06-18T13:02:36.469888+0000","last_active":"2024-06-18T13:03:07.420699+0000","last_peered":"2024-06-18T13:03:07.420699+0000","last_clean":"2024-06-18T13:03:07.420699+0000","last_became_active":"2024-06-18T13:02:36.469269+0000","last_became_peered":"2024-06-18T13:02:36.469269+0000","last_unstale":"2024-06-18T13:03:07.420699+0000","last_undegraded":"2024-06-18T13:03:07.420699+0000","last_fullsized":"2024-06-18T13:03:07.420699+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T22:52:31.531988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"69'10","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.423411+0000","last_change":"2024-06-18T13:02:32.412282+0000","last_active":"2024-06-18T13:03:07.423411+0000","last_peered":"2024-06-18T13:03:07.423411+0000","last_clean":"2024-06-18T13:03:07.423411+0000","last_became_active":"2024-06-18T13:02:32.411958+0000","last_became_peered":"2024-06-18T13:02:32.411958+0000","last_unstale":"2024-06-18T13:03:07.423411+0000","last_undegraded":"2024-06-18T13:03:07.423411+0000","last_fullsized":"2024-06-18T13:03:07.423411+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:54:17.816941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.421804+0000","last_change":"2024-06-18T13:02:30.412493+0000","last_active":"2024-06-18T13:03:07.421804+0000","last_peered":"2024-06-18T13:03:07.421804+0000","last_clean":"2024-06-18T13:03:07.421804+0000","last_became_active":"2024-06-18T13:02:30.412192+0000","last_became_peered":"2024-06-18T13:02:30.412192+0000","last_unstale":"2024-06-18T13:03:07.421804+0000","last_undegraded":"2024-06-18T13:03:07.421804+0000","last_fullsized":"2024-06-18T13:03:07.421804+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:04:53.921051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"69'8","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:40.474314+0000","last_change":"2024-06-18T13:02:34.438824+0000","last_active":"2024-06-18T13:03:40.474314+0000","last_peered":"2024-06-18T13:03:40.474314+0000","last_clean":"2024-06-18T13:03:40.474314+0000","last_became_active":"2024-06-18T13:02:34.438505+0000","last_became_peered":"2024-06-18T13:02:34.438505+0000","last_unstale":"2024-06-18T13:03:40.474314+0000","last_undegraded":"2024-06-18T13:03:40.474314+0000","last_fullsized":"2024-06-18T13:03:40.474314+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:12:02.840238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443073+0000","last_change":"2024-06-18T13:02:36.452331+0000","last_active":"2024-06-18T13:03:07.443073+0000","last_peered":"2024-06-18T13:03:07.443073+0000","last_clean":"2024-06-18T13:03:07.443073+0000","last_became_active":"2024-06-18T13:02:36.452030+0000","last_became_peered":"2024-06-18T13:02:36.452030+0000","last_unstale":"2024-06-18T13:03:07.443073+0000","last_undegraded":"2024-06-18T13:03:07.443073+0000","last_fullsized":"2024-06-18T13:03:07.443073+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:29:50.986345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"69'28","reported_seq":89,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:40.474327+0000","last_change":"2024-06-18T13:02:32.421366+0000","last_active":"2024-06-18T13:03:40.474327+0000","last_peered":"2024-06-18T13:03:40.474327+0000","last_clean":"2024-06-18T13:03:40.474327+0000","last_became_active":"2024-06-18T13:02:32.421039+0000","last_became_peered":"2024-06-18T13:02:32.421039+0000","last_unstale":"2024-06-18T13:03:40.474327+0000","last_undegraded":"2024-06-18T13:03:40.474327+0000","last_fullsized":"2024-06-18T13:03:40.474327+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:14:02.563774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444232+0000","last_change":"2024-06-18T13:02:30.412383+0000","last_active":"2024-06-18T13:03:07.444232+0000","last_peered":"2024-06-18T13:03:07.444232+0000","last_clean":"2024-06-18T13:03:07.444232+0000","last_became_active":"2024-06-18T13:02:30.412097+0000","last_became_peered":"2024-06-18T13:02:30.412097+0000","last_unstale":"2024-06-18T13:03:07.444232+0000","last_undegraded":"2024-06-18T13:03:07.444232+0000","last_fullsized":"2024-06-18T13:03:07.444232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:48:14.289434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443673+0000","last_change":"2024-06-18T13:02:34.432634+0000","last_active":"2024-06-18T13:03:07.443673+0000","last_peered":"2024-06-18T13:03:07.443673+0000","last_clean":"2024-06-18T13:03:07.443673+0000","last_became_active":"2024-06-18T13:02:34.432341+0000","last_became_peered":"2024-06-18T13:02:34.432341+0000","last_unstale":"2024-06-18T13:03:07.443673+0000","last_undegraded":"2024-06-18T13:03:07.443673+0000","last_fullsized":"2024-06-18T13:03:07.443673+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T22:46:19.795026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420729+0000","last_change":"2024-06-18T13:02:36.451323+0000","last_active":"2024-06-18T13:03:07.420729+0000","last_peered":"2024-06-18T13:03:07.420729+0000","last_clean":"2024-06-18T13:03:07.420729+0000","last_became_active":"2024-06-18T13:02:36.450594+0000","last_became_peered":"2024-06-18T13:02:36.450594+0000","last_unstale":"2024-06-18T13:03:07.420729+0000","last_undegraded":"2024-06-18T13:03:07.420729+0000","last_fullsized":"2024-06-18T13:03:07.420729+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:44:23.737894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"69'13","reported_seq":52,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.423279+0000","last_change":"2024-06-18T13:02:32.417395+0000","last_active":"2024-06-18T13:03:07.423279+0000","last_peered":"2024-06-18T13:03:07.423279+0000","last_clean":"2024-06-18T13:03:07.423279+0000","last_became_active":"2024-06-18T13:02:32.417100+0000","last_became_peered":"2024-06-18T13:02:32.417100+0000","last_unstale":"2024-06-18T13:03:07.423279+0000","last_undegraded":"2024-06-18T13:03:07.423279+0000","last_fullsized":"2024-06-18T13:03:07.423279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:23:10.045343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"69'5","reported_seq":110,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:46.526737+0000","last_change":"2024-06-18T13:02:34.434651+0000","last_active":"2024-06-18T13:03:46.526737+0000","last_peered":"2024-06-18T13:03:46.526737+0000","last_clean":"2024-06-18T13:03:46.526737+0000","last_became_active":"2024-06-18T13:02:30.415191+0000","last_became_peered":"2024-06-18T13:02:30.415191+0000","last_unstale":"2024-06-18T13:03:46.526737+0000","last_undegraded":"2024-06-18T13:03:46.526737+0000","last_fullsized":"2024-06-18T13:03:46.526737+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:45:40.393421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089436150000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":71,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443390+0000","last_change":"2024-06-18T13:02:30.413876+0000","last_active":"2024-06-18T13:03:07.443390+0000","last_peered":"2024-06-18T13:03:07.443390+0000","last_clean":"2024-06-18T13:03:07.443390+0000","last_became_active":"2024-06-18T13:02:30.413558+0000","last_became_peered":"2024-06-18T13:02:30.413558+0000","last_unstale":"2024-06-18T13:03:07.443390+0000","last_undegraded":"2024-06-18T13:03:07.443390+0000","last_fullsized":"2024-06-18T13:03:07.443390+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:54:10.596757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443940+0000","last_change":"2024-06-18T13:02:34.435612+0000","last_active":"2024-06-18T13:03:07.443940+0000","last_peered":"2024-06-18T13:03:07.443940+0000","last_clean":"2024-06-18T13:03:07.443940+0000","last_became_active":"2024-06-18T13:02:34.435181+0000","last_became_peered":"2024-06-18T13:02:34.435181+0000","last_unstale":"2024-06-18T13:03:07.443940+0000","last_undegraded":"2024-06-18T13:03:07.443940+0000","last_fullsized":"2024-06-18T13:03:07.443940+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:23:20.217222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.418210+0000","last_change":"2024-06-18T13:02:36.469708+0000","last_active":"2024-06-18T13:03:07.418210+0000","last_peered":"2024-06-18T13:03:07.418210+0000","last_clean":"2024-06-18T13:03:07.418210+0000","last_became_active":"2024-06-18T13:02:36.469428+0000","last_became_peered":"2024-06-18T13:02:36.469428+0000","last_unstale":"2024-06-18T13:03:07.418210+0000","last_undegraded":"2024-06-18T13:03:07.418210+0000","last_fullsized":"2024-06-18T13:03:07.418210+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:46:54.170335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"69'12","reported_seq":43,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444596+0000","last_change":"2024-06-18T13:02:32.419047+0000","last_active":"2024-06-18T13:03:07.444596+0000","last_peered":"2024-06-18T13:03:07.444596+0000","last_clean":"2024-06-18T13:03:07.444596+0000","last_became_active":"2024-06-18T13:02:32.418558+0000","last_became_peered":"2024-06-18T13:02:32.418558+0000","last_unstale":"2024-06-18T13:03:07.444596+0000","last_undegraded":"2024-06-18T13:03:07.444596+0000","last_fullsized":"2024-06-18T13:03:07.444596+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:15:55.070054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":36,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443539+0000","last_change":"2024-06-18T13:02:34.424339+0000","last_active":"2024-06-18T13:03:07.443539+0000","last_peered":"2024-06-18T13:03:07.443539+0000","last_clean":"2024-06-18T13:03:07.443539+0000","last_became_active":"2024-06-18T13:02:30.417572+0000","last_became_peered":"2024-06-18T13:02:30.417572+0000","last_unstale":"2024-06-18T13:03:07.443539+0000","last_undegraded":"2024-06-18T13:03:07.443539+0000","last_fullsized":"2024-06-18T13:03:07.443539+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:36:03.626819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050054199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417600+0000","last_change":"2024-06-18T13:02:30.420941+0000","last_active":"2024-06-18T13:03:07.417600+0000","last_peered":"2024-06-18T13:03:07.417600+0000","last_clean":"2024-06-18T13:03:07.417600+0000","last_became_active":"2024-06-18T13:02:30.420700+0000","last_became_peered":"2024-06-18T13:02:30.420700+0000","last_unstale":"2024-06-18T13:03:07.417600+0000","last_undegraded":"2024-06-18T13:03:07.417600+0000","last_fullsized":"2024-06-18T13:03:07.417600+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:28:06.339022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445195+0000","last_change":"2024-06-18T13:02:34.439543+0000","last_active":"2024-06-18T13:03:07.445195+0000","last_peered":"2024-06-18T13:03:07.445195+0000","last_clean":"2024-06-18T13:03:07.445195+0000","last_became_active":"2024-06-18T13:02:34.438871+0000","last_became_peered":"2024-06-18T13:02:34.438871+0000","last_unstale":"2024-06-18T13:03:07.445195+0000","last_undegraded":"2024-06-18T13:03:07.445195+0000","last_fullsized":"2024-06-18T13:03:07.445195+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:18:39.026087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.421380+0000","last_change":"2024-06-18T13:02:36.451108+0000","last_active":"2024-06-18T13:03:07.421380+0000","last_peered":"2024-06-18T13:03:07.421380+0000","last_clean":"2024-06-18T13:03:07.421380+0000","last_became_active":"2024-06-18T13:02:36.450771+0000","last_became_peered":"2024-06-18T13:02:36.450771+0000","last_unstale":"2024-06-18T13:03:07.421380+0000","last_undegraded":"2024-06-18T13:03:07.421380+0000","last_fullsized":"2024-06-18T13:03:07.421380+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:57:16.812917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"69'16","reported_seq":63,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:40.474528+0000","last_change":"2024-06-18T13:02:32.533948+0000","last_active":"2024-06-18T13:03:40.474528+0000","last_peered":"2024-06-18T13:03:40.474528+0000","last_clean":"2024-06-18T13:03:40.474528+0000","last_became_active":"2024-06-18T13:02:32.533648+0000","last_became_peered":"2024-06-18T13:02:32.533648+0000","last_unstale":"2024-06-18T13:03:40.474528+0000","last_undegraded":"2024-06-18T13:03:40.474528+0000","last_fullsized":"2024-06-18T13:03:40.474528+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:20:47.152859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"66'2","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419602+0000","last_change":"2024-06-18T13:02:34.434675+0000","last_active":"2024-06-18T13:03:07.419602+0000","last_peered":"2024-06-18T13:03:07.419602+0000","last_clean":"2024-06-18T13:03:07.419602+0000","last_became_active":"2024-06-18T13:02:30.418683+0000","last_became_peered":"2024-06-18T13:02:30.418683+0000","last_unstale":"2024-06-18T13:03:07.419602+0000","last_undegraded":"2024-06-18T13:03:07.419602+0000","last_fullsized":"2024-06-18T13:03:07.419602+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:23:17.212638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027193109999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444493+0000","last_change":"2024-06-18T13:02:30.413336+0000","last_active":"2024-06-18T13:03:07.444493+0000","last_peered":"2024-06-18T13:03:07.444493+0000","last_clean":"2024-06-18T13:03:07.444493+0000","last_became_active":"2024-06-18T13:02:30.412973+0000","last_became_peered":"2024-06-18T13:02:30.412973+0000","last_unstale":"2024-06-18T13:03:07.444493+0000","last_undegraded":"2024-06-18T13:03:07.444493+0000","last_fullsized":"2024-06-18T13:03:07.444493+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:54:27.312192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":275,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:09.499371+0000","last_change":"2024-06-18T13:02:18.639016+0000","last_active":"2024-06-18T13:03:09.499371+0000","last_peered":"2024-06-18T13:03:09.499371+0000","last_clean":"2024-06-18T13:03:09.499371+0000","last_became_active":"2024-06-18T13:02:18.638579+0000","last_became_peered":"2024-06-18T13:02:18.638579+0000","last_unstale":"2024-06-18T13:03:09.499371+0000","last_undegraded":"2024-06-18T13:03:09.499371+0000","last_fullsized":"2024-06-18T13:03:09.499371+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:00:22.175291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:00:22.175291+0000","last_clean_scrub_stamp":"2024-06-18T13:00:22.175291+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:24:19.503964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.418033+0000","last_change":"2024-06-18T13:02:34.442649+0000","last_active":"2024-06-18T13:03:07.418033+0000","last_peered":"2024-06-18T13:03:07.418033+0000","last_clean":"2024-06-18T13:03:07.418033+0000","last_became_active":"2024-06-18T13:02:34.442349+0000","last_became_peered":"2024-06-18T13:02:34.442349+0000","last_unstale":"2024-06-18T13:03:07.418033+0000","last_undegraded":"2024-06-18T13:03:07.418033+0000","last_fullsized":"2024-06-18T13:03:07.418033+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:14:17.239342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445250+0000","last_change":"2024-06-18T13:02:36.446367+0000","last_active":"2024-06-18T13:03:07.445250+0000","last_peered":"2024-06-18T13:03:07.445250+0000","last_clean":"2024-06-18T13:03:07.445250+0000","last_became_active":"2024-06-18T13:02:36.446060+0000","last_became_peered":"2024-06-18T13:02:36.446060+0000","last_unstale":"2024-06-18T13:03:07.445250+0000","last_undegraded":"2024-06-18T13:03:07.445250+0000","last_fullsized":"2024-06-18T13:03:07.445250+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T22:10:50.569169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"69'19","reported_seq":56,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443414+0000","last_change":"2024-06-18T13:02:32.533548+0000","last_active":"2024-06-18T13:03:07.443414+0000","last_peered":"2024-06-18T13:03:07.443414+0000","last_clean":"2024-06-18T13:03:07.443414+0000","last_became_active":"2024-06-18T13:02:32.533224+0000","last_became_peered":"2024-06-18T13:02:32.533224+0000","last_unstale":"2024-06-18T13:03:07.443414+0000","last_undegraded":"2024-06-18T13:03:07.443414+0000","last_fullsized":"2024-06-18T13:03:07.443414+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:30:29.678869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443915+0000","last_change":"2024-06-18T13:02:30.421233+0000","last_active":"2024-06-18T13:03:07.443915+0000","last_peered":"2024-06-18T13:03:07.443915+0000","last_clean":"2024-06-18T13:03:07.443915+0000","last_became_active":"2024-06-18T13:02:30.420935+0000","last_became_peered":"2024-06-18T13:02:30.420935+0000","last_unstale":"2024-06-18T13:03:07.443915+0000","last_undegraded":"2024-06-18T13:03:07.443915+0000","last_fullsized":"2024-06-18T13:03:07.443915+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:47:11.104838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443464+0000","last_change":"2024-06-18T13:02:34.435847+0000","last_active":"2024-06-18T13:03:07.443464+0000","last_peered":"2024-06-18T13:03:07.443464+0000","last_clean":"2024-06-18T13:03:07.443464+0000","last_became_active":"2024-06-18T13:02:34.435545+0000","last_became_peered":"2024-06-18T13:02:34.435545+0000","last_unstale":"2024-06-18T13:03:07.443464+0000","last_undegraded":"2024-06-18T13:03:07.443464+0000","last_fullsized":"2024-06-18T13:03:07.443464+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:22:04.151349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"69'1","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417680+0000","last_change":"2024-06-18T13:02:36.451501+0000","last_active":"2024-06-18T13:03:07.417680+0000","last_peered":"2024-06-18T13:03:07.417680+0000","last_clean":"2024-06-18T13:03:07.417680+0000","last_became_active":"2024-06-18T13:02:36.451215+0000","last_became_peered":"2024-06-18T13:02:36.451215+0000","last_unstale":"2024-06-18T13:03:07.417680+0000","last_undegraded":"2024-06-18T13:03:07.417680+0000","last_fullsized":"2024-06-18T13:03:07.417680+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:21:42.922203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"69'9","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443499+0000","last_change":"2024-06-18T13:02:32.419605+0000","last_active":"2024-06-18T13:03:07.443499+0000","last_peered":"2024-06-18T13:03:07.443499+0000","last_clean":"2024-06-18T13:03:07.443499+0000","last_became_active":"2024-06-18T13:02:32.419275+0000","last_became_peered":"2024-06-18T13:02:32.419275+0000","last_unstale":"2024-06-18T13:03:07.443499+0000","last_undegraded":"2024-06-18T13:03:07.443499+0000","last_fullsized":"2024-06-18T13:03:07.443499+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:21:14.031837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.422462+0000","last_change":"2024-06-18T13:02:30.414122+0000","last_active":"2024-06-18T13:03:07.422462+0000","last_peered":"2024-06-18T13:03:07.422462+0000","last_clean":"2024-06-18T13:03:07.422462+0000","last_became_active":"2024-06-18T13:02:30.413848+0000","last_became_peered":"2024-06-18T13:02:30.413848+0000","last_unstale":"2024-06-18T13:03:07.422462+0000","last_undegraded":"2024-06-18T13:03:07.422462+0000","last_fullsized":"2024-06-18T13:03:07.422462+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:39:51.884607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443607+0000","last_change":"2024-06-18T13:02:34.435744+0000","last_active":"2024-06-18T13:03:07.443607+0000","last_peered":"2024-06-18T13:03:07.443607+0000","last_clean":"2024-06-18T13:03:07.443607+0000","last_became_active":"2024-06-18T13:02:34.435441+0000","last_became_peered":"2024-06-18T13:02:34.435441+0000","last_unstale":"2024-06-18T13:03:07.443607+0000","last_undegraded":"2024-06-18T13:03:07.443607+0000","last_fullsized":"2024-06-18T13:03:07.443607+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:38:46.429243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443448+0000","last_change":"2024-06-18T13:02:36.448885+0000","last_active":"2024-06-18T13:03:07.443448+0000","last_peered":"2024-06-18T13:03:07.443448+0000","last_clean":"2024-06-18T13:03:07.443448+0000","last_became_active":"2024-06-18T13:02:36.448529+0000","last_became_peered":"2024-06-18T13:02:36.448529+0000","last_unstale":"2024-06-18T13:03:07.443448+0000","last_undegraded":"2024-06-18T13:03:07.443448+0000","last_fullsized":"2024-06-18T13:03:07.443448+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:01:21.884620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"69'15","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443227+0000","last_change":"2024-06-18T13:02:32.534570+0000","last_active":"2024-06-18T13:03:07.443227+0000","last_peered":"2024-06-18T13:03:07.443227+0000","last_clean":"2024-06-18T13:03:07.443227+0000","last_became_active":"2024-06-18T13:02:32.534103+0000","last_became_peered":"2024-06-18T13:02:32.534103+0000","last_unstale":"2024-06-18T13:03:07.443227+0000","last_undegraded":"2024-06-18T13:03:07.443227+0000","last_fullsized":"2024-06-18T13:03:07.443227+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:50:17.718171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444082+0000","last_change":"2024-06-18T13:02:30.417488+0000","last_active":"2024-06-18T13:03:07.444082+0000","last_peered":"2024-06-18T13:03:07.444082+0000","last_clean":"2024-06-18T13:03:07.444082+0000","last_became_active":"2024-06-18T13:02:30.416850+0000","last_became_peered":"2024-06-18T13:02:30.416850+0000","last_unstale":"2024-06-18T13:03:07.444082+0000","last_undegraded":"2024-06-18T13:03:07.444082+0000","last_fullsized":"2024-06-18T13:03:07.444082+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:00:24.082688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"69'8","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:40.474577+0000","last_change":"2024-06-18T13:02:34.437602+0000","last_active":"2024-06-18T13:03:40.474577+0000","last_peered":"2024-06-18T13:03:40.474577+0000","last_clean":"2024-06-18T13:03:40.474577+0000","last_became_active":"2024-06-18T13:02:34.437067+0000","last_became_peered":"2024-06-18T13:02:34.437067+0000","last_unstale":"2024-06-18T13:03:40.474577+0000","last_undegraded":"2024-06-18T13:03:40.474577+0000","last_fullsized":"2024-06-18T13:03:40.474577+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:53:26.262004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443859+0000","last_change":"2024-06-18T13:02:36.469535+0000","last_active":"2024-06-18T13:03:07.443859+0000","last_peered":"2024-06-18T13:03:07.443859+0000","last_clean":"2024-06-18T13:03:07.443859+0000","last_became_active":"2024-06-18T13:02:36.469236+0000","last_became_peered":"2024-06-18T13:02:36.469236+0000","last_unstale":"2024-06-18T13:03:07.443859+0000","last_undegraded":"2024-06-18T13:03:07.443859+0000","last_fullsized":"2024-06-18T13:03:07.443859+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:13:11.671125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"69'12","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443579+0000","last_change":"2024-06-18T13:02:32.418020+0000","last_active":"2024-06-18T13:03:07.443579+0000","last_peered":"2024-06-18T13:03:07.443579+0000","last_clean":"2024-06-18T13:03:07.443579+0000","last_became_active":"2024-06-18T13:02:32.417586+0000","last_became_peered":"2024-06-18T13:02:32.417586+0000","last_unstale":"2024-06-18T13:03:07.443579+0000","last_undegraded":"2024-06-18T13:03:07.443579+0000","last_fullsized":"2024-06-18T13:03:07.443579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:24:49.047854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"62'2","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445325+0000","last_change":"2024-06-18T13:02:30.415409+0000","last_active":"2024-06-18T13:03:07.445325+0000","last_peered":"2024-06-18T13:03:07.445325+0000","last_clean":"2024-06-18T13:03:07.445325+0000","last_became_active":"2024-06-18T13:02:30.415071+0000","last_became_peered":"2024-06-18T13:02:30.415071+0000","last_unstale":"2024-06-18T13:03:07.445325+0000","last_undegraded":"2024-06-18T13:03:07.445325+0000","last_fullsized":"2024-06-18T13:03:07.445325+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:51:42.249337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443084+0000","last_change":"2024-06-18T13:02:34.433724+0000","last_active":"2024-06-18T13:03:07.443084+0000","last_peered":"2024-06-18T13:03:07.443084+0000","last_clean":"2024-06-18T13:03:07.443084+0000","last_became_active":"2024-06-18T13:02:34.433352+0000","last_became_peered":"2024-06-18T13:02:34.433352+0000","last_unstale":"2024-06-18T13:03:07.443084+0000","last_undegraded":"2024-06-18T13:03:07.443084+0000","last_fullsized":"2024-06-18T13:03:07.443084+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:34:38.238250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419707+0000","last_change":"2024-06-18T13:02:36.452292+0000","last_active":"2024-06-18T13:03:07.419707+0000","last_peered":"2024-06-18T13:03:07.419707+0000","last_clean":"2024-06-18T13:03:07.419707+0000","last_became_active":"2024-06-18T13:02:36.450576+0000","last_became_peered":"2024-06-18T13:02:36.450576+0000","last_unstale":"2024-06-18T13:03:07.419707+0000","last_undegraded":"2024-06-18T13:03:07.419707+0000","last_fullsized":"2024-06-18T13:03:07.419707+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:50:24.609427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"69'9","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444532+0000","last_change":"2024-06-18T13:02:32.533559+0000","last_active":"2024-06-18T13:03:07.444532+0000","last_peered":"2024-06-18T13:03:07.444532+0000","last_clean":"2024-06-18T13:03:07.444532+0000","last_became_active":"2024-06-18T13:02:32.533271+0000","last_became_peered":"2024-06-18T13:02:32.533271+0000","last_unstale":"2024-06-18T13:03:07.444532+0000","last_undegraded":"2024-06-18T13:03:07.444532+0000","last_fullsized":"2024-06-18T13:03:07.444532+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:07:55.627893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443090+0000","last_change":"2024-06-18T13:02:30.412180+0000","last_active":"2024-06-18T13:03:07.443090+0000","last_peered":"2024-06-18T13:03:07.443090+0000","last_clean":"2024-06-18T13:03:07.443090+0000","last_became_active":"2024-06-18T13:02:30.411887+0000","last_became_peered":"2024-06-18T13:02:30.411887+0000","last_unstale":"2024-06-18T13:03:07.443090+0000","last_undegraded":"2024-06-18T13:03:07.443090+0000","last_fullsized":"2024-06-18T13:03:07.443090+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:47:52.143479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420472+0000","last_change":"2024-06-18T13:02:34.446713+0000","last_active":"2024-06-18T13:03:07.420472+0000","last_peered":"2024-06-18T13:03:07.420472+0000","last_clean":"2024-06-18T13:03:07.420472+0000","last_became_active":"2024-06-18T13:02:34.446455+0000","last_became_peered":"2024-06-18T13:02:34.446455+0000","last_unstale":"2024-06-18T13:03:07.420472+0000","last_undegraded":"2024-06-18T13:03:07.420472+0000","last_fullsized":"2024-06-18T13:03:07.420472+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:47:23.223424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443409+0000","last_change":"2024-06-18T13:02:36.444328+0000","last_active":"2024-06-18T13:03:07.443409+0000","last_peered":"2024-06-18T13:03:07.443409+0000","last_clean":"2024-06-18T13:03:07.443409+0000","last_became_active":"2024-06-18T13:02:36.443639+0000","last_became_peered":"2024-06-18T13:02:36.443639+0000","last_unstale":"2024-06-18T13:03:07.443409+0000","last_undegraded":"2024-06-18T13:03:07.443409+0000","last_fullsized":"2024-06-18T13:03:07.443409+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:58:08.219150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443728+0000","last_change":"2024-06-18T13:02:30.416751+0000","last_active":"2024-06-18T13:03:07.443728+0000","last_peered":"2024-06-18T13:03:07.443728+0000","last_clean":"2024-06-18T13:03:07.443728+0000","last_became_active":"2024-06-18T13:02:30.416493+0000","last_became_peered":"2024-06-18T13:02:30.416493+0000","last_unstale":"2024-06-18T13:03:07.443728+0000","last_undegraded":"2024-06-18T13:03:07.443728+0000","last_fullsized":"2024-06-18T13:03:07.443728+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:05:33.797355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"69'6","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420560+0000","last_change":"2024-06-18T13:02:32.534357+0000","last_active":"2024-06-18T13:03:07.420560+0000","last_peered":"2024-06-18T13:03:07.420560+0000","last_clean":"2024-06-18T13:03:07.420560+0000","last_became_active":"2024-06-18T13:02:32.534070+0000","last_became_peered":"2024-06-18T13:02:32.534070+0000","last_unstale":"2024-06-18T13:03:07.420560+0000","last_undegraded":"2024-06-18T13:03:07.420560+0000","last_fullsized":"2024-06-18T13:03:07.420560+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:42:53.361894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444965+0000","last_change":"2024-06-18T13:02:34.441909+0000","last_active":"2024-06-18T13:03:07.444965+0000","last_peered":"2024-06-18T13:03:07.444965+0000","last_clean":"2024-06-18T13:03:07.444965+0000","last_became_active":"2024-06-18T13:02:34.441543+0000","last_became_peered":"2024-06-18T13:02:34.441543+0000","last_unstale":"2024-06-18T13:03:07.444965+0000","last_undegraded":"2024-06-18T13:03:07.444965+0000","last_fullsized":"2024-06-18T13:03:07.444965+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T22:10:20.863847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.418121+0000","last_change":"2024-06-18T13:02:36.471949+0000","last_active":"2024-06-18T13:03:07.418121+0000","last_peered":"2024-06-18T13:03:07.418121+0000","last_clean":"2024-06-18T13:03:07.418121+0000","last_became_active":"2024-06-18T13:02:36.471644+0000","last_became_peered":"2024-06-18T13:02:36.471644+0000","last_unstale":"2024-06-18T13:03:07.418121+0000","last_undegraded":"2024-06-18T13:03:07.418121+0000","last_fullsized":"2024-06-18T13:03:07.418121+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:15:22.042120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"69'10","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420328+0000","last_change":"2024-06-18T13:02:32.533884+0000","last_active":"2024-06-18T13:03:07.420328+0000","last_peered":"2024-06-18T13:03:07.420328+0000","last_clean":"2024-06-18T13:03:07.420328+0000","last_became_active":"2024-06-18T13:02:32.533545+0000","last_became_peered":"2024-06-18T13:02:32.533545+0000","last_unstale":"2024-06-18T13:03:07.420328+0000","last_undegraded":"2024-06-18T13:03:07.420328+0000","last_fullsized":"2024-06-18T13:03:07.420328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:04:33.758753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444196+0000","last_change":"2024-06-18T13:02:30.414245+0000","last_active":"2024-06-18T13:03:07.444196+0000","last_peered":"2024-06-18T13:03:07.444196+0000","last_clean":"2024-06-18T13:03:07.444196+0000","last_became_active":"2024-06-18T13:02:30.413920+0000","last_became_peered":"2024-06-18T13:02:30.413920+0000","last_unstale":"2024-06-18T13:03:07.444196+0000","last_undegraded":"2024-06-18T13:03:07.444196+0000","last_fullsized":"2024-06-18T13:03:07.444196+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:51:02.772980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"69'8","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:40.474419+0000","last_change":"2024-06-18T13:02:34.440909+0000","last_active":"2024-06-18T13:03:40.474419+0000","last_peered":"2024-06-18T13:03:40.474419+0000","last_clean":"2024-06-18T13:03:40.474419+0000","last_became_active":"2024-06-18T13:02:34.440538+0000","last_became_peered":"2024-06-18T13:02:34.440538+0000","last_unstale":"2024-06-18T13:03:40.474419+0000","last_undegraded":"2024-06-18T13:03:40.474419+0000","last_fullsized":"2024-06-18T13:03:40.474419+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:12:34.973776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444300+0000","last_change":"2024-06-18T13:02:36.454638+0000","last_active":"2024-06-18T13:03:07.444300+0000","last_peered":"2024-06-18T13:03:07.444300+0000","last_clean":"2024-06-18T13:03:07.444300+0000","last_became_active":"2024-06-18T13:02:36.454294+0000","last_became_peered":"2024-06-18T13:02:36.454294+0000","last_unstale":"2024-06-18T13:03:07.444300+0000","last_undegraded":"2024-06-18T13:03:07.444300+0000","last_fullsized":"2024-06-18T13:03:07.444300+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:05:25.333139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"69'9","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444981+0000","last_change":"2024-06-18T13:02:32.533679+0000","last_active":"2024-06-18T13:03:07.444981+0000","last_peered":"2024-06-18T13:03:07.444981+0000","last_clean":"2024-06-18T13:03:07.444981+0000","last_became_active":"2024-06-18T13:02:32.533358+0000","last_became_peered":"2024-06-18T13:02:32.533358+0000","last_unstale":"2024-06-18T13:03:07.444981+0000","last_undegraded":"2024-06-18T13:03:07.444981+0000","last_fullsized":"2024-06-18T13:03:07.444981+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:03:07.091045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443819+0000","last_change":"2024-06-18T13:02:30.415519+0000","last_active":"2024-06-18T13:03:07.443819+0000","last_peered":"2024-06-18T13:03:07.443819+0000","last_clean":"2024-06-18T13:03:07.443819+0000","last_became_active":"2024-06-18T13:02:30.414709+0000","last_became_peered":"2024-06-18T13:02:30.414709+0000","last_unstale":"2024-06-18T13:03:07.443819+0000","last_undegraded":"2024-06-18T13:03:07.443819+0000","last_fullsized":"2024-06-18T13:03:07.443819+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:32:28.153975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"69'8","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:40.474403+0000","last_change":"2024-06-18T13:02:34.447600+0000","last_active":"2024-06-18T13:03:40.474403+0000","last_peered":"2024-06-18T13:03:40.474403+0000","last_clean":"2024-06-18T13:03:40.474403+0000","last_became_active":"2024-06-18T13:02:34.447251+0000","last_became_peered":"2024-06-18T13:02:34.447251+0000","last_unstale":"2024-06-18T13:03:40.474403+0000","last_undegraded":"2024-06-18T13:03:40.474403+0000","last_fullsized":"2024-06-18T13:03:40.474403+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:54:11.312071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445084+0000","last_change":"2024-06-18T13:02:36.445621+0000","last_active":"2024-06-18T13:03:07.445084+0000","last_peered":"2024-06-18T13:03:07.445084+0000","last_clean":"2024-06-18T13:03:07.445084+0000","last_became_active":"2024-06-18T13:02:36.445281+0000","last_became_peered":"2024-06-18T13:02:36.445281+0000","last_unstale":"2024-06-18T13:03:07.445084+0000","last_undegraded":"2024-06-18T13:03:07.445084+0000","last_fullsized":"2024-06-18T13:03:07.445084+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:35:37.750819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"69'9","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.423371+0000","last_change":"2024-06-18T13:02:32.415035+0000","last_active":"2024-06-18T13:03:07.423371+0000","last_peered":"2024-06-18T13:03:07.423371+0000","last_clean":"2024-06-18T13:03:07.423371+0000","last_became_active":"2024-06-18T13:02:32.414539+0000","last_became_peered":"2024-06-18T13:02:32.414539+0000","last_unstale":"2024-06-18T13:03:07.423371+0000","last_undegraded":"2024-06-18T13:03:07.423371+0000","last_fullsized":"2024-06-18T13:03:07.423371+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:11:20.956605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444183+0000","last_change":"2024-06-18T13:02:30.414277+0000","last_active":"2024-06-18T13:03:07.444183+0000","last_peered":"2024-06-18T13:03:07.444183+0000","last_clean":"2024-06-18T13:03:07.444183+0000","last_became_active":"2024-06-18T13:02:30.414007+0000","last_became_peered":"2024-06-18T13:02:30.414007+0000","last_unstale":"2024-06-18T13:03:07.444183+0000","last_undegraded":"2024-06-18T13:03:07.444183+0000","last_fullsized":"2024-06-18T13:03:07.444183+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:27:49.237415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419798+0000","last_change":"2024-06-18T13:02:34.438273+0000","last_active":"2024-06-18T13:03:07.419798+0000","last_peered":"2024-06-18T13:03:07.419798+0000","last_clean":"2024-06-18T13:03:07.419798+0000","last_became_active":"2024-06-18T13:02:34.437964+0000","last_became_peered":"2024-06-18T13:02:34.437964+0000","last_unstale":"2024-06-18T13:03:07.419798+0000","last_undegraded":"2024-06-18T13:03:07.419798+0000","last_fullsized":"2024-06-18T13:03:07.419798+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:59:42.364269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443051+0000","last_change":"2024-06-18T13:02:36.453525+0000","last_active":"2024-06-18T13:03:07.443051+0000","last_peered":"2024-06-18T13:03:07.443051+0000","last_clean":"2024-06-18T13:03:07.443051+0000","last_became_active":"2024-06-18T13:02:36.453239+0000","last_became_peered":"2024-06-18T13:02:36.453239+0000","last_unstale":"2024-06-18T13:03:07.443051+0000","last_undegraded":"2024-06-18T13:03:07.443051+0000","last_fullsized":"2024-06-18T13:03:07.443051+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:35:26.563010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"69'11","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444608+0000","last_change":"2024-06-18T13:02:32.414219+0000","last_active":"2024-06-18T13:03:07.444608+0000","last_peered":"2024-06-18T13:03:07.444608+0000","last_clean":"2024-06-18T13:03:07.444608+0000","last_became_active":"2024-06-18T13:02:32.413885+0000","last_became_peered":"2024-06-18T13:02:32.413885+0000","last_unstale":"2024-06-18T13:03:07.444608+0000","last_undegraded":"2024-06-18T13:03:07.444608+0000","last_fullsized":"2024-06-18T13:03:07.444608+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-20T00:51:00.963524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.422385+0000","last_change":"2024-06-18T13:02:30.407418+0000","last_active":"2024-06-18T13:03:07.422385+0000","last_peered":"2024-06-18T13:03:07.422385+0000","last_clean":"2024-06-18T13:03:07.422385+0000","last_became_active":"2024-06-18T13:02:30.406648+0000","last_became_peered":"2024-06-18T13:02:30.406648+0000","last_unstale":"2024-06-18T13:03:07.422385+0000","last_undegraded":"2024-06-18T13:03:07.422385+0000","last_fullsized":"2024-06-18T13:03:07.422385+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:16:32.062110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.423228+0000","last_change":"2024-06-18T13:02:34.443388+0000","last_active":"2024-06-18T13:03:07.423228+0000","last_peered":"2024-06-18T13:03:07.423228+0000","last_clean":"2024-06-18T13:03:07.423228+0000","last_became_active":"2024-06-18T13:02:34.442490+0000","last_became_peered":"2024-06-18T13:02:34.442490+0000","last_unstale":"2024-06-18T13:03:07.423228+0000","last_undegraded":"2024-06-18T13:03:07.423228+0000","last_fullsized":"2024-06-18T13:03:07.423228+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:19:17.974739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419207+0000","last_change":"2024-06-18T13:02:36.453644+0000","last_active":"2024-06-18T13:03:07.419207+0000","last_peered":"2024-06-18T13:03:07.419207+0000","last_clean":"2024-06-18T13:03:07.419207+0000","last_became_active":"2024-06-18T13:02:36.453301+0000","last_became_peered":"2024-06-18T13:02:36.453301+0000","last_unstale":"2024-06-18T13:03:07.419207+0000","last_undegraded":"2024-06-18T13:03:07.419207+0000","last_fullsized":"2024-06-18T13:03:07.419207+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:20:49.347688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"69'4","reported_seq":31,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419236+0000","last_change":"2024-06-18T13:02:32.422818+0000","last_active":"2024-06-18T13:03:07.419236+0000","last_peered":"2024-06-18T13:03:07.419236+0000","last_clean":"2024-06-18T13:03:07.419236+0000","last_became_active":"2024-06-18T13:02:32.422429+0000","last_became_peered":"2024-06-18T13:02:32.422429+0000","last_unstale":"2024-06-18T13:03:07.419236+0000","last_undegraded":"2024-06-18T13:03:07.419236+0000","last_fullsized":"2024-06-18T13:03:07.419236+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:59:22.912024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"62'1","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445056+0000","last_change":"2024-06-18T13:02:30.414551+0000","last_active":"2024-06-18T13:03:07.445056+0000","last_peered":"2024-06-18T13:03:07.445056+0000","last_clean":"2024-06-18T13:03:07.445056+0000","last_became_active":"2024-06-18T13:02:30.414253+0000","last_became_peered":"2024-06-18T13:02:30.414253+0000","last_unstale":"2024-06-18T13:03:07.445056+0000","last_undegraded":"2024-06-18T13:03:07.445056+0000","last_fullsized":"2024-06-18T13:03:07.445056+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:37:38.599058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443549+0000","last_change":"2024-06-18T13:02:34.442532+0000","last_active":"2024-06-18T13:03:07.443549+0000","last_peered":"2024-06-18T13:03:07.443549+0000","last_clean":"2024-06-18T13:03:07.443549+0000","last_became_active":"2024-06-18T13:02:34.442084+0000","last_became_peered":"2024-06-18T13:02:34.442084+0000","last_unstale":"2024-06-18T13:03:07.443549+0000","last_undegraded":"2024-06-18T13:03:07.443549+0000","last_fullsized":"2024-06-18T13:03:07.443549+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-20T00:19:31.477775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417486+0000","last_change":"2024-06-18T13:02:36.448715+0000","last_active":"2024-06-18T13:03:07.417486+0000","last_peered":"2024-06-18T13:03:07.417486+0000","last_clean":"2024-06-18T13:03:07.417486+0000","last_became_active":"2024-06-18T13:02:36.448371+0000","last_became_peered":"2024-06-18T13:02:36.448371+0000","last_unstale":"2024-06-18T13:03:07.417486+0000","last_undegraded":"2024-06-18T13:03:07.417486+0000","last_fullsized":"2024-06-18T13:03:07.417486+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:30:18.503424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"69'11","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419565+0000","last_change":"2024-06-18T13:02:32.534315+0000","last_active":"2024-06-18T13:03:07.419565+0000","last_peered":"2024-06-18T13:03:07.419565+0000","last_clean":"2024-06-18T13:03:07.419565+0000","last_became_active":"2024-06-18T13:02:32.533868+0000","last_became_peered":"2024-06-18T13:02:32.533868+0000","last_unstale":"2024-06-18T13:03:07.419565+0000","last_undegraded":"2024-06-18T13:03:07.419565+0000","last_fullsized":"2024-06-18T13:03:07.419565+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T22:44:00.755969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444114+0000","last_change":"2024-06-18T13:02:30.412351+0000","last_active":"2024-06-18T13:03:07.444114+0000","last_peered":"2024-06-18T13:03:07.444114+0000","last_clean":"2024-06-18T13:03:07.444114+0000","last_became_active":"2024-06-18T13:02:30.412067+0000","last_became_peered":"2024-06-18T13:02:30.412067+0000","last_unstale":"2024-06-18T13:03:07.444114+0000","last_undegraded":"2024-06-18T13:03:07.444114+0000","last_fullsized":"2024-06-18T13:03:07.444114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:45:04.636876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417301+0000","last_change":"2024-06-18T13:02:34.445850+0000","last_active":"2024-06-18T13:03:07.417301+0000","last_peered":"2024-06-18T13:03:07.417301+0000","last_clean":"2024-06-18T13:03:07.417301+0000","last_became_active":"2024-06-18T13:02:34.445524+0000","last_became_peered":"2024-06-18T13:02:34.445524+0000","last_unstale":"2024-06-18T13:03:07.417301+0000","last_undegraded":"2024-06-18T13:03:07.417301+0000","last_fullsized":"2024-06-18T13:03:07.417301+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:26:06.694880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419612+0000","last_change":"2024-06-18T13:02:36.471298+0000","last_active":"2024-06-18T13:03:07.419612+0000","last_peered":"2024-06-18T13:03:07.419612+0000","last_clean":"2024-06-18T13:03:07.419612+0000","last_became_active":"2024-06-18T13:02:36.470978+0000","last_became_peered":"2024-06-18T13:02:36.470978+0000","last_unstale":"2024-06-18T13:03:07.419612+0000","last_undegraded":"2024-06-18T13:03:07.419612+0000","last_fullsized":"2024-06-18T13:03:07.419612+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:23:17.370819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"69'1","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417846+0000","last_change":"2024-06-18T13:02:36.448818+0000","last_active":"2024-06-18T13:03:07.417846+0000","last_peered":"2024-06-18T13:03:07.417846+0000","last_clean":"2024-06-18T13:03:07.417846+0000","last_became_active":"2024-06-18T13:02:36.448532+0000","last_became_peered":"2024-06-18T13:02:36.448532+0000","last_unstale":"2024-06-18T13:03:07.417846+0000","last_undegraded":"2024-06-18T13:03:07.417846+0000","last_fullsized":"2024-06-18T13:03:07.417846+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:52:07.803432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443385+0000","last_change":"2024-06-18T13:02:30.411673+0000","last_active":"2024-06-18T13:03:07.443385+0000","last_peered":"2024-06-18T13:03:07.443385+0000","last_clean":"2024-06-18T13:03:07.443385+0000","last_became_active":"2024-06-18T13:02:30.411369+0000","last_became_peered":"2024-06-18T13:02:30.411369+0000","last_unstale":"2024-06-18T13:03:07.443385+0000","last_undegraded":"2024-06-18T13:03:07.443385+0000","last_fullsized":"2024-06-18T13:03:07.443385+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:17:58.513431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"69'10","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444516+0000","last_change":"2024-06-18T13:02:32.533612+0000","last_active":"2024-06-18T13:03:07.444516+0000","last_peered":"2024-06-18T13:03:07.444516+0000","last_clean":"2024-06-18T13:03:07.444516+0000","last_became_active":"2024-06-18T13:02:32.533351+0000","last_became_peered":"2024-06-18T13:02:32.533351+0000","last_unstale":"2024-06-18T13:03:07.444516+0000","last_undegraded":"2024-06-18T13:03:07.444516+0000","last_fullsized":"2024-06-18T13:03:07.444516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:19:26.903661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"69'8","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:40.474391+0000","last_change":"2024-06-18T13:02:34.443478+0000","last_active":"2024-06-18T13:03:40.474391+0000","last_peered":"2024-06-18T13:03:40.474391+0000","last_clean":"2024-06-18T13:03:40.474391+0000","last_became_active":"2024-06-18T13:02:34.442263+0000","last_became_peered":"2024-06-18T13:02:34.442263+0000","last_unstale":"2024-06-18T13:03:40.474391+0000","last_undegraded":"2024-06-18T13:03:40.474391+0000","last_fullsized":"2024-06-18T13:03:40.474391+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:09:15.038131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.421918+0000","last_change":"2024-06-18T13:02:36.443442+0000","last_active":"2024-06-18T13:03:07.421918+0000","last_peered":"2024-06-18T13:03:07.421918+0000","last_clean":"2024-06-18T13:03:07.421918+0000","last_became_active":"2024-06-18T13:02:36.443133+0000","last_became_peered":"2024-06-18T13:02:36.443133+0000","last_unstale":"2024-06-18T13:03:07.421918+0000","last_undegraded":"2024-06-18T13:03:07.421918+0000","last_fullsized":"2024-06-18T13:03:07.421918+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:36:51.652527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"62'1","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420006+0000","last_change":"2024-06-18T13:02:30.419427+0000","last_active":"2024-06-18T13:03:07.420006+0000","last_peered":"2024-06-18T13:03:07.420006+0000","last_clean":"2024-06-18T13:03:07.420006+0000","last_became_active":"2024-06-18T13:02:30.418293+0000","last_became_peered":"2024-06-18T13:02:30.418293+0000","last_unstale":"2024-06-18T13:03:07.420006+0000","last_undegraded":"2024-06-18T13:03:07.420006+0000","last_fullsized":"2024-06-18T13:03:07.420006+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:45:43.288433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"69'11","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444062+0000","last_change":"2024-06-18T13:02:32.422917+0000","last_active":"2024-06-18T13:03:07.444062+0000","last_peered":"2024-06-18T13:03:07.444062+0000","last_clean":"2024-06-18T13:03:07.444062+0000","last_became_active":"2024-06-18T13:02:32.422619+0000","last_became_peered":"2024-06-18T13:02:32.422619+0000","last_unstale":"2024-06-18T13:03:07.444062+0000","last_undegraded":"2024-06-18T13:03:07.444062+0000","last_fullsized":"2024-06-18T13:03:07.444062+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:51:17.241337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444434+0000","last_change":"2024-06-18T13:02:34.438808+0000","last_active":"2024-06-18T13:03:07.444434+0000","last_peered":"2024-06-18T13:03:07.444434+0000","last_clean":"2024-06-18T13:03:07.444434+0000","last_became_active":"2024-06-18T13:02:34.438478+0000","last_became_peered":"2024-06-18T13:02:34.438478+0000","last_unstale":"2024-06-18T13:03:07.444434+0000","last_undegraded":"2024-06-18T13:03:07.444434+0000","last_fullsized":"2024-06-18T13:03:07.444434+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T22:30:48.205792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444971+0000","last_change":"2024-06-18T13:02:36.470203+0000","last_active":"2024-06-18T13:03:07.444971+0000","last_peered":"2024-06-18T13:03:07.444971+0000","last_clean":"2024-06-18T13:03:07.444971+0000","last_became_active":"2024-06-18T13:02:36.469810+0000","last_became_peered":"2024-06-18T13:02:36.469810+0000","last_unstale":"2024-06-18T13:03:07.444971+0000","last_undegraded":"2024-06-18T13:03:07.444971+0000","last_fullsized":"2024-06-18T13:03:07.444971+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:26:15.264159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443680+0000","last_change":"2024-06-18T13:02:30.411860+0000","last_active":"2024-06-18T13:03:07.443680+0000","last_peered":"2024-06-18T13:03:07.443680+0000","last_clean":"2024-06-18T13:03:07.443680+0000","last_became_active":"2024-06-18T13:02:30.411592+0000","last_became_peered":"2024-06-18T13:02:30.411592+0000","last_unstale":"2024-06-18T13:03:07.443680+0000","last_undegraded":"2024-06-18T13:03:07.443680+0000","last_fullsized":"2024-06-18T13:03:07.443680+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:14:57.880958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"69'15","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443669+0000","last_change":"2024-06-18T13:02:32.416596+0000","last_active":"2024-06-18T13:03:07.443669+0000","last_peered":"2024-06-18T13:03:07.443669+0000","last_clean":"2024-06-18T13:03:07.443669+0000","last_became_active":"2024-06-18T13:02:32.416275+0000","last_became_peered":"2024-06-18T13:02:32.416275+0000","last_unstale":"2024-06-18T13:03:07.443669+0000","last_undegraded":"2024-06-18T13:03:07.443669+0000","last_fullsized":"2024-06-18T13:03:07.443669+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:35:49.192197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.422189+0000","last_change":"2024-06-18T13:02:34.429610+0000","last_active":"2024-06-18T13:03:07.422189+0000","last_peered":"2024-06-18T13:03:07.422189+0000","last_clean":"2024-06-18T13:03:07.422189+0000","last_became_active":"2024-06-18T13:02:34.429268+0000","last_became_peered":"2024-06-18T13:02:34.429268+0000","last_unstale":"2024-06-18T13:03:07.422189+0000","last_undegraded":"2024-06-18T13:03:07.422189+0000","last_fullsized":"2024-06-18T13:03:07.422189+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:13:31.374986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420270+0000","last_change":"2024-06-18T13:02:36.471398+0000","last_active":"2024-06-18T13:03:07.420270+0000","last_peered":"2024-06-18T13:03:07.420270+0000","last_clean":"2024-06-18T13:03:07.420270+0000","last_became_active":"2024-06-18T13:02:36.471038+0000","last_became_peered":"2024-06-18T13:02:36.471038+0000","last_unstale":"2024-06-18T13:03:07.420270+0000","last_undegraded":"2024-06-18T13:03:07.420270+0000","last_fullsized":"2024-06-18T13:03:07.420270+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:30:41.530082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419924+0000","last_change":"2024-06-18T13:02:30.419110+0000","last_active":"2024-06-18T13:03:07.419924+0000","last_peered":"2024-06-18T13:03:07.419924+0000","last_clean":"2024-06-18T13:03:07.419924+0000","last_became_active":"2024-06-18T13:02:30.418827+0000","last_became_peered":"2024-06-18T13:02:30.418827+0000","last_unstale":"2024-06-18T13:03:07.419924+0000","last_undegraded":"2024-06-18T13:03:07.419924+0000","last_fullsized":"2024-06-18T13:03:07.419924+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:26:13.772453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"69'12","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420077+0000","last_change":"2024-06-18T13:02:32.423756+0000","last_active":"2024-06-18T13:03:07.420077+0000","last_peered":"2024-06-18T13:03:07.420077+0000","last_clean":"2024-06-18T13:03:07.420077+0000","last_became_active":"2024-06-18T13:02:32.423449+0000","last_became_peered":"2024-06-18T13:02:32.423449+0000","last_unstale":"2024-06-18T13:03:07.420077+0000","last_undegraded":"2024-06-18T13:03:07.420077+0000","last_fullsized":"2024-06-18T13:03:07.420077+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:50:28.551618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444731+0000","last_change":"2024-06-18T13:02:34.439676+0000","last_active":"2024-06-18T13:03:07.444731+0000","last_peered":"2024-06-18T13:03:07.444731+0000","last_clean":"2024-06-18T13:03:07.444731+0000","last_became_active":"2024-06-18T13:02:34.439253+0000","last_became_peered":"2024-06-18T13:02:34.439253+0000","last_unstale":"2024-06-18T13:03:07.444731+0000","last_undegraded":"2024-06-18T13:03:07.444731+0000","last_fullsized":"2024-06-18T13:03:07.444731+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:03:14.794520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":66,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":56,"seq":240518168598,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32532,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712620,"statfs":{"total":95995035648,"available":95961722880,"internally_reserved":0,"allocated":1388544,"data_stored":916249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462938,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32528,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712624,"statfs":{"total":95995035648,"available":95961726976,"internally_reserved":0,"allocated":1384448,"data_stored":917014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724575,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31940,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713212,"statfs":{"total":95995035648,"available":95962329088,"internally_reserved":0,"allocated":778240,"data_stored":326088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":794624,"data_stored":325295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149736,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":815104,"data_stored":325659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27784,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717368,"statfs":{"total":95995035648,"available":95966584832,"internally_reserved":0,"allocated":782336,"data_stored":326715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":827392,"data_stored":326456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712620,"statfs":{"total":95995035648,"available":95961722880,"internally_reserved":0,"allocated":1384448,"data_stored":915509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-18T13:03:53.804 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph pg dump --format=json 2024-06-18T13:03:54.043 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:54 smithi167 ceph-mon[37701]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-18T13:03:54.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:54 smithi167 ceph-mon[29496]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-18T13:03:54.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:54 smithi179 ceph-mon[33312]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-18T13:03:55.146 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:03:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:03:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:03:55.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:55 smithi167 ceph-mon[37701]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:55.457 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:55 smithi167 ceph-mon[29496]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:55.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.631+0000 7fb8481f1640 1 -- 172.21.15.167:0/1030943909 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb840102e90 msgr2=0x7fb840107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:55.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.631+0000 7fb8481f1640 1 --2- 172.21.15.167:0/1030943909 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb840102e90 0x7fb840107340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb830009860 tx=0x7fb83002a340 comp rx=0 tx=0).stop 2024-06-18T13:03:55.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.631+0000 7fb8481f1640 1 -- 172.21.15.167:0/1030943909 shutdown_connections 2024-06-18T13:03:55.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.631+0000 7fb8481f1640 1 --2- 172.21.15.167:0/1030943909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb840107a50 0x7fb84010e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:55.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.631+0000 7fb8481f1640 1 --2- 172.21.15.167:0/1030943909 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb840102e90 0x7fb840107340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:55.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.631+0000 7fb8481f1640 1 --2- 172.21.15.167:0/1030943909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8401024e0 0x7fb8401028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:55.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.631+0000 7fb8481f1640 1 -- 172.21.15.167:0/1030943909 >> 172.21.15.167:0/1030943909 conn(0x7fb8400fbf80 msgr2=0x7fb8400fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:55.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.631+0000 7fb8481f1640 1 -- 172.21.15.167:0/1030943909 shutdown_connections 2024-06-18T13:03:55.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.631+0000 7fb8481f1640 1 -- 172.21.15.167:0/1030943909 wait complete. 2024-06-18T13:03:55.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.631+0000 7fb8481f1640 1 Processor -- start 2024-06-18T13:03:55.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.631+0000 7fb8481f1640 1 -- start start 2024-06-18T13:03:55.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb8481f1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8401024e0 0x7fb8401972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:55.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb8481f1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb840102e90 0x7fb840197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:55.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb8481f1640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb840107a50 0x7fb84019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:55.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb8481f1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb840110c80 con 0x7fb8401024e0 2024-06-18T13:03:55.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb8481f1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb840110b00 con 0x7fb840102e90 2024-06-18T13:03:55.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb8481f1640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb840110e00 con 0x7fb840107a50 2024-06-18T13:03:55.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb845f8d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8401024e0 0x7fb8401972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:55.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb84678e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb840107a50 0x7fb84019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:55.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb845f8d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8401024e0 0x7fb8401972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:36364/0 (socket says 172.21.15.167:36364) 2024-06-18T13:03:55.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb845f8d640 1 -- 172.21.15.167:0/1652159450 learned_addr learned my addr 172.21.15.167:0/1652159450 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:55.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb84578c640 1 --2- 172.21.15.167:0/1652159450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb840102e90 0x7fb840197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:55.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb845f8d640 1 -- 172.21.15.167:0/1652159450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb840107a50 msgr2=0x7fb84019bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:55.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb845f8d640 1 --2- 172.21.15.167:0/1652159450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb840107a50 0x7fb84019bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:55.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb845f8d640 1 -- 172.21.15.167:0/1652159450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb840102e90 msgr2=0x7fb840197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:55.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb845f8d640 1 --2- 172.21.15.167:0/1652159450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb840102e90 0x7fb840197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:55.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb845f8d640 1 -- 172.21.15.167:0/1652159450 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84019c280 con 0x7fb8401024e0 2024-06-18T13:03:55.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb84578c640 1 --2- 172.21.15.167:0/1652159450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb840102e90 0x7fb840197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T13:03:55.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb845f8d640 1 --2- 172.21.15.167:0/1652159450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8401024e0 0x7fb8401972d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00e9c0 tx=0x7fb83c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:55.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.632+0000 7fb836ffd640 1 -- 172.21.15.167:0/1652159450 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c00cde0 con 0x7fb8401024e0 2024-06-18T13:03:55.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.633+0000 7fb836ffd640 1 -- 172.21.15.167:0/1652159450 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fb83c0035f0 con 0x7fb8401024e0 2024-06-18T13:03:55.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.633+0000 7fb8481f1640 1 -- 172.21.15.167:0/1652159450 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84019c570 con 0x7fb8401024e0 2024-06-18T13:03:55.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.633+0000 7fb836ffd640 1 -- 172.21.15.167:0/1652159450 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0044a0 con 0x7fb8401024e0 2024-06-18T13:03:55.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.633+0000 7fb8481f1640 1 -- 172.21.15.167:0/1652159450 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb840100070 con 0x7fb8401024e0 2024-06-18T13:03:55.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.633+0000 7fb8481f1640 1 -- 172.21.15.167:0/1652159450 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8401028c0 con 0x7fb8401024e0 2024-06-18T13:03:55.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.634+0000 7fb836ffd640 1 -- 172.21.15.167:0/1652159450 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7fb83c004000 con 0x7fb8401024e0 2024-06-18T13:03:55.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.635+0000 7fb836ffd640 1 --2- 172.21.15.167:0/1652159450 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb82c06e040 0x7fb82c070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:55.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.635+0000 7fb836ffd640 1 -- 172.21.15.167:0/1652159450 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7fb83c08e970 con 0x7fb8401024e0 2024-06-18T13:03:55.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.635+0000 7fb84578c640 1 --2- 172.21.15.167:0/1652159450 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb82c06e040 0x7fb82c070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:55.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.635+0000 7fb84578c640 1 --2- 172.21.15.167:0/1652159450 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb82c06e040 0x7fb82c070500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb830035040 tx=0x7fb8300305d0 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:55.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:55 smithi179 ceph-mon[33312]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:55.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.639+0000 7fb836ffd640 1 -- 172.21.15.167:0/1652159450 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fb83c05bdf0 con 0x7fb8401024e0 2024-06-18T13:03:55.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.809+0000 7fb8481f1640 1 -- 172.21.15.167:0/1652159450 --> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb840106f50 con 0x7fb82c06e040 2024-06-18T13:03:55.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.817+0000 7fb836ffd640 1 -- 172.21.15.167:0/1652159450 <== mgr.24461 v2:172.21.15.167:6800/2104657861 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346474 (secure 0 0 0) 0x7fb840106f50 con 0x7fb82c06e040 2024-06-18T13:03:55.820 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T13:03:55.821 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-06-18T13:03:55.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.820+0000 7fb8481f1640 1 -- 172.21.15.167:0/1652159450 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb82c06e040 msgr2=0x7fb82c070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:55.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.820+0000 7fb8481f1640 1 --2- 172.21.15.167:0/1652159450 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb82c06e040 0x7fb82c070500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb830035040 tx=0x7fb8300305d0 comp rx=0 tx=0).stop 2024-06-18T13:03:55.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.820+0000 7fb8481f1640 1 -- 172.21.15.167:0/1652159450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8401024e0 msgr2=0x7fb8401972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:55.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.820+0000 7fb8481f1640 1 --2- 172.21.15.167:0/1652159450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8401024e0 0x7fb8401972d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00e9c0 tx=0x7fb83c00ee80 comp rx=0 tx=0).stop 2024-06-18T13:03:55.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.820+0000 7fb8481f1640 1 -- 172.21.15.167:0/1652159450 shutdown_connections 2024-06-18T13:03:55.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.820+0000 7fb8481f1640 1 --2- 172.21.15.167:0/1652159450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb840107a50 0x7fb84019bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:55.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.820+0000 7fb8481f1640 1 --2- 172.21.15.167:0/1652159450 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb82c06e040 0x7fb82c070500 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:55.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.820+0000 7fb8481f1640 1 --2- 172.21.15.167:0/1652159450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb840102e90 0x7fb840197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:55.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.820+0000 7fb8481f1640 1 --2- 172.21.15.167:0/1652159450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8401024e0 0x7fb8401972d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:55.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.820+0000 7fb8481f1640 1 -- 172.21.15.167:0/1652159450 >> 172.21.15.167:0/1652159450 conn(0x7fb8400fbf80 msgr2=0x7fb840105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:55.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.821+0000 7fb8481f1640 1 -- 172.21.15.167:0/1652159450 shutdown_connections 2024-06-18T13:03:55.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:55.821+0000 7fb8481f1640 1 -- 172.21.15.167:0/1652159450 wait complete. 2024-06-18T13:03:56.486 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-06-18T13:03:55.440091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1010,"num_read_kb":856,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":249076,"kb_used_data":7964,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":749712140,"statfs":{"total":767960285184,"available":767705231360,"internally_reserved":0,"allocated":8155136,"data_stored":4378985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002820"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419508+0000","last_change":"2024-06-18T13:02:36.472243+0000","last_active":"2024-06-18T13:03:07.419508+0000","last_peered":"2024-06-18T13:03:07.419508+0000","last_clean":"2024-06-18T13:03:07.419508+0000","last_became_active":"2024-06-18T13:02:36.471928+0000","last_became_peered":"2024-06-18T13:02:36.471928+0000","last_unstale":"2024-06-18T13:03:07.419508+0000","last_undegraded":"2024-06-18T13:03:07.419508+0000","last_fullsized":"2024-06-18T13:03:07.419508+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:12:16.543101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444127+0000","last_change":"2024-06-18T13:02:30.418875+0000","last_active":"2024-06-18T13:03:07.444127+0000","last_peered":"2024-06-18T13:03:07.444127+0000","last_clean":"2024-06-18T13:03:07.444127+0000","last_became_active":"2024-06-18T13:02:30.418554+0000","last_became_peered":"2024-06-18T13:02:30.418554+0000","last_unstale":"2024-06-18T13:03:07.444127+0000","last_undegraded":"2024-06-18T13:03:07.444127+0000","last_fullsized":"2024-06-18T13:03:07.444127+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:22:32.818017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"69'15","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419757+0000","last_change":"2024-06-18T13:02:32.416501+0000","last_active":"2024-06-18T13:03:07.419757+0000","last_peered":"2024-06-18T13:03:07.419757+0000","last_clean":"2024-06-18T13:03:07.419757+0000","last_became_active":"2024-06-18T13:02:32.416134+0000","last_became_peered":"2024-06-18T13:02:32.416134+0000","last_unstale":"2024-06-18T13:03:07.419757+0000","last_undegraded":"2024-06-18T13:03:07.419757+0000","last_fullsized":"2024-06-18T13:03:07.419757+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:21:05.663556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443946+0000","last_change":"2024-06-18T13:02:34.437890+0000","last_active":"2024-06-18T13:03:07.443946+0000","last_peered":"2024-06-18T13:03:07.443946+0000","last_clean":"2024-06-18T13:03:07.443946+0000","last_became_active":"2024-06-18T13:02:34.437560+0000","last_became_peered":"2024-06-18T13:02:34.437560+0000","last_unstale":"2024-06-18T13:03:07.443946+0000","last_undegraded":"2024-06-18T13:03:07.443946+0000","last_fullsized":"2024-06-18T13:03:07.443946+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:32:57.541440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420865+0000","last_change":"2024-06-18T13:02:30.418269+0000","last_active":"2024-06-18T13:03:07.420865+0000","last_peered":"2024-06-18T13:03:07.420865+0000","last_clean":"2024-06-18T13:03:07.420865+0000","last_became_active":"2024-06-18T13:02:30.417989+0000","last_became_peered":"2024-06-18T13:02:30.417989+0000","last_unstale":"2024-06-18T13:03:07.420865+0000","last_undegraded":"2024-06-18T13:03:07.420865+0000","last_fullsized":"2024-06-18T13:03:07.420865+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:31:22.346990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"69'9","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443971+0000","last_change":"2024-06-18T13:02:32.414812+0000","last_active":"2024-06-18T13:03:07.443971+0000","last_peered":"2024-06-18T13:03:07.443971+0000","last_clean":"2024-06-18T13:03:07.443971+0000","last_became_active":"2024-06-18T13:02:32.414482+0000","last_became_peered":"2024-06-18T13:02:32.414482+0000","last_unstale":"2024-06-18T13:03:07.443971+0000","last_undegraded":"2024-06-18T13:03:07.443971+0000","last_fullsized":"2024-06-18T13:03:07.443971+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:40:53.728503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420781+0000","last_change":"2024-06-18T13:02:34.434587+0000","last_active":"2024-06-18T13:03:07.420781+0000","last_peered":"2024-06-18T13:03:07.420781+0000","last_clean":"2024-06-18T13:03:07.420781+0000","last_became_active":"2024-06-18T13:02:34.434292+0000","last_became_peered":"2024-06-18T13:02:34.434292+0000","last_unstale":"2024-06-18T13:03:07.420781+0000","last_undegraded":"2024-06-18T13:03:07.420781+0000","last_fullsized":"2024-06-18T13:03:07.420781+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:35:57.259405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444334+0000","last_change":"2024-06-18T13:02:36.446931+0000","last_active":"2024-06-18T13:03:07.444334+0000","last_peered":"2024-06-18T13:03:07.444334+0000","last_clean":"2024-06-18T13:03:07.444334+0000","last_became_active":"2024-06-18T13:02:36.446627+0000","last_became_peered":"2024-06-18T13:02:36.446627+0000","last_unstale":"2024-06-18T13:03:07.444334+0000","last_undegraded":"2024-06-18T13:03:07.444334+0000","last_fullsized":"2024-06-18T13:03:07.444334+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:53:21.698408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.418484+0000","last_change":"2024-06-18T13:02:30.419267+0000","last_active":"2024-06-18T13:03:07.418484+0000","last_peered":"2024-06-18T13:03:07.418484+0000","last_clean":"2024-06-18T13:03:07.418484+0000","last_became_active":"2024-06-18T13:02:30.419007+0000","last_became_peered":"2024-06-18T13:02:30.419007+0000","last_unstale":"2024-06-18T13:03:07.418484+0000","last_undegraded":"2024-06-18T13:03:07.418484+0000","last_fullsized":"2024-06-18T13:03:07.418484+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:28:59.680188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"69'5","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444548+0000","last_change":"2024-06-18T13:02:32.417988+0000","last_active":"2024-06-18T13:03:07.444548+0000","last_peered":"2024-06-18T13:03:07.444548+0000","last_clean":"2024-06-18T13:03:07.444548+0000","last_became_active":"2024-06-18T13:02:32.417687+0000","last_became_peered":"2024-06-18T13:02:32.417687+0000","last_unstale":"2024-06-18T13:03:07.444548+0000","last_undegraded":"2024-06-18T13:03:07.444548+0000","last_fullsized":"2024-06-18T13:03:07.444548+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:39:46.131680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417463+0000","last_change":"2024-06-18T13:02:34.439133+0000","last_active":"2024-06-18T13:03:07.417463+0000","last_peered":"2024-06-18T13:03:07.417463+0000","last_clean":"2024-06-18T13:03:07.417463+0000","last_became_active":"2024-06-18T13:02:34.438258+0000","last_became_peered":"2024-06-18T13:02:34.438258+0000","last_unstale":"2024-06-18T13:03:07.417463+0000","last_undegraded":"2024-06-18T13:03:07.417463+0000","last_fullsized":"2024-06-18T13:03:07.417463+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:42:21.415594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443611+0000","last_change":"2024-06-18T13:02:36.451296+0000","last_active":"2024-06-18T13:03:07.443611+0000","last_peered":"2024-06-18T13:03:07.443611+0000","last_clean":"2024-06-18T13:03:07.443611+0000","last_became_active":"2024-06-18T13:02:36.450940+0000","last_became_peered":"2024-06-18T13:02:36.450940+0000","last_unstale":"2024-06-18T13:03:07.443611+0000","last_undegraded":"2024-06-18T13:03:07.443611+0000","last_fullsized":"2024-06-18T13:03:07.443611+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:27:56.156544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.418897+0000","last_change":"2024-06-18T13:02:30.423190+0000","last_active":"2024-06-18T13:03:07.418897+0000","last_peered":"2024-06-18T13:03:07.418897+0000","last_clean":"2024-06-18T13:03:07.418897+0000","last_became_active":"2024-06-18T13:02:30.422916+0000","last_became_peered":"2024-06-18T13:02:30.422916+0000","last_unstale":"2024-06-18T13:03:07.418897+0000","last_undegraded":"2024-06-18T13:03:07.418897+0000","last_fullsized":"2024-06-18T13:03:07.418897+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:00:27.122463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"69'9","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444493+0000","last_change":"2024-06-18T13:02:32.417902+0000","last_active":"2024-06-18T13:03:07.444493+0000","last_peered":"2024-06-18T13:03:07.444493+0000","last_clean":"2024-06-18T13:03:07.444493+0000","last_became_active":"2024-06-18T13:02:32.417504+0000","last_became_peered":"2024-06-18T13:02:32.417504+0000","last_unstale":"2024-06-18T13:03:07.444493+0000","last_undegraded":"2024-06-18T13:03:07.444493+0000","last_fullsized":"2024-06-18T13:03:07.444493+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:00:03.673111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444365+0000","last_change":"2024-06-18T13:02:34.441891+0000","last_active":"2024-06-18T13:03:07.444365+0000","last_peered":"2024-06-18T13:03:07.444365+0000","last_clean":"2024-06-18T13:03:07.444365+0000","last_became_active":"2024-06-18T13:02:34.441474+0000","last_became_peered":"2024-06-18T13:02:34.441474+0000","last_unstale":"2024-06-18T13:03:07.444365+0000","last_undegraded":"2024-06-18T13:03:07.444365+0000","last_fullsized":"2024-06-18T13:03:07.444365+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:56:11.258359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443192+0000","last_change":"2024-06-18T13:02:36.452515+0000","last_active":"2024-06-18T13:03:07.443192+0000","last_peered":"2024-06-18T13:03:07.443192+0000","last_clean":"2024-06-18T13:03:07.443192+0000","last_became_active":"2024-06-18T13:02:36.452215+0000","last_became_peered":"2024-06-18T13:02:36.452215+0000","last_unstale":"2024-06-18T13:03:07.443192+0000","last_undegraded":"2024-06-18T13:03:07.443192+0000","last_fullsized":"2024-06-18T13:03:07.443192+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:36:31.678457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"69'17","reported_seq":53,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445059+0000","last_change":"2024-06-18T13:02:32.415203+0000","last_active":"2024-06-18T13:03:07.445059+0000","last_peered":"2024-06-18T13:03:07.445059+0000","last_clean":"2024-06-18T13:03:07.445059+0000","last_became_active":"2024-06-18T13:02:32.414900+0000","last_became_peered":"2024-06-18T13:02:32.414900+0000","last_unstale":"2024-06-18T13:03:07.445059+0000","last_undegraded":"2024-06-18T13:03:07.445059+0000","last_fullsized":"2024-06-18T13:03:07.445059+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:18:51.613696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.418154+0000","last_change":"2024-06-18T13:02:30.418466+0000","last_active":"2024-06-18T13:03:07.418154+0000","last_peered":"2024-06-18T13:03:07.418154+0000","last_clean":"2024-06-18T13:03:07.418154+0000","last_became_active":"2024-06-18T13:02:30.418168+0000","last_became_peered":"2024-06-18T13:02:30.418168+0000","last_unstale":"2024-06-18T13:03:07.418154+0000","last_undegraded":"2024-06-18T13:03:07.418154+0000","last_fullsized":"2024-06-18T13:03:07.418154+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:57:56.080381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.421467+0000","last_change":"2024-06-18T13:02:34.429611+0000","last_active":"2024-06-18T13:03:07.421467+0000","last_peered":"2024-06-18T13:03:07.421467+0000","last_clean":"2024-06-18T13:03:07.421467+0000","last_became_active":"2024-06-18T13:02:34.429344+0000","last_became_peered":"2024-06-18T13:02:34.429344+0000","last_unstale":"2024-06-18T13:03:07.421467+0000","last_undegraded":"2024-06-18T13:03:07.421467+0000","last_fullsized":"2024-06-18T13:03:07.421467+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:24:38.969355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443682+0000","last_change":"2024-06-18T13:02:36.444742+0000","last_active":"2024-06-18T13:03:07.443682+0000","last_peered":"2024-06-18T13:03:07.443682+0000","last_clean":"2024-06-18T13:03:07.443682+0000","last_became_active":"2024-06-18T13:02:36.444411+0000","last_became_peered":"2024-06-18T13:02:36.444411+0000","last_unstale":"2024-06-18T13:03:07.443682+0000","last_undegraded":"2024-06-18T13:03:07.443682+0000","last_fullsized":"2024-06-18T13:03:07.443682+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:07:18.359065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"69'10","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444814+0000","last_change":"2024-06-18T13:02:32.418114+0000","last_active":"2024-06-18T13:03:07.444814+0000","last_peered":"2024-06-18T13:03:07.444814+0000","last_clean":"2024-06-18T13:03:07.444814+0000","last_became_active":"2024-06-18T13:02:32.417669+0000","last_became_peered":"2024-06-18T13:02:32.417669+0000","last_unstale":"2024-06-18T13:03:07.444814+0000","last_undegraded":"2024-06-18T13:03:07.444814+0000","last_fullsized":"2024-06-18T13:03:07.444814+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:12:28.539021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.423542+0000","last_change":"2024-06-18T13:02:30.413373+0000","last_active":"2024-06-18T13:03:07.423542+0000","last_peered":"2024-06-18T13:03:07.423542+0000","last_clean":"2024-06-18T13:03:07.423542+0000","last_became_active":"2024-06-18T13:02:30.412688+0000","last_became_peered":"2024-06-18T13:02:30.412688+0000","last_unstale":"2024-06-18T13:03:07.423542+0000","last_undegraded":"2024-06-18T13:03:07.423542+0000","last_fullsized":"2024-06-18T13:03:07.423542+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T22:34:19.272376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"69'8","reported_seq":46,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:45.475569+0000","last_change":"2024-06-18T13:02:34.436308+0000","last_active":"2024-06-18T13:03:45.475569+0000","last_peered":"2024-06-18T13:03:45.475569+0000","last_clean":"2024-06-18T13:03:45.475569+0000","last_became_active":"2024-06-18T13:02:34.436033+0000","last_became_peered":"2024-06-18T13:02:34.436033+0000","last_unstale":"2024-06-18T13:03:45.475569+0000","last_undegraded":"2024-06-18T13:03:45.475569+0000","last_fullsized":"2024-06-18T13:03:45.475569+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-20T01:02:01.872653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445414+0000","last_change":"2024-06-18T13:02:36.446884+0000","last_active":"2024-06-18T13:03:07.445414+0000","last_peered":"2024-06-18T13:03:07.445414+0000","last_clean":"2024-06-18T13:03:07.445414+0000","last_became_active":"2024-06-18T13:02:36.446567+0000","last_became_peered":"2024-06-18T13:02:36.446567+0000","last_unstale":"2024-06-18T13:03:07.445414+0000","last_undegraded":"2024-06-18T13:03:07.445414+0000","last_fullsized":"2024-06-18T13:03:07.445414+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-20T00:48:04.981730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"69'15","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.423150+0000","last_change":"2024-06-18T13:02:32.421344+0000","last_active":"2024-06-18T13:03:07.423150+0000","last_peered":"2024-06-18T13:03:07.423150+0000","last_clean":"2024-06-18T13:03:07.423150+0000","last_became_active":"2024-06-18T13:02:32.420960+0000","last_became_peered":"2024-06-18T13:02:32.420960+0000","last_unstale":"2024-06-18T13:03:07.423150+0000","last_undegraded":"2024-06-18T13:03:07.423150+0000","last_fullsized":"2024-06-18T13:03:07.423150+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-20T01:00:53.325751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420976+0000","last_change":"2024-06-18T13:02:30.413272+0000","last_active":"2024-06-18T13:03:07.420976+0000","last_peered":"2024-06-18T13:03:07.420976+0000","last_clean":"2024-06-18T13:03:07.420976+0000","last_became_active":"2024-06-18T13:02:30.412656+0000","last_became_peered":"2024-06-18T13:02:30.412656+0000","last_unstale":"2024-06-18T13:03:07.420976+0000","last_undegraded":"2024-06-18T13:03:07.420976+0000","last_fullsized":"2024-06-18T13:03:07.420976+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:46:18.761210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"69'8","reported_seq":46,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:45.475626+0000","last_change":"2024-06-18T13:02:34.436572+0000","last_active":"2024-06-18T13:03:45.475626+0000","last_peered":"2024-06-18T13:03:45.475626+0000","last_clean":"2024-06-18T13:03:45.475626+0000","last_became_active":"2024-06-18T13:02:34.435825+0000","last_became_peered":"2024-06-18T13:02:34.435825+0000","last_unstale":"2024-06-18T13:03:45.475626+0000","last_undegraded":"2024-06-18T13:03:45.475626+0000","last_fullsized":"2024-06-18T13:03:45.475626+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:57:01.564982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444674+0000","last_change":"2024-06-18T13:02:36.447260+0000","last_active":"2024-06-18T13:03:07.444674+0000","last_peered":"2024-06-18T13:03:07.444674+0000","last_clean":"2024-06-18T13:03:07.444674+0000","last_became_active":"2024-06-18T13:02:36.446977+0000","last_became_peered":"2024-06-18T13:02:36.446977+0000","last_unstale":"2024-06-18T13:03:07.444674+0000","last_undegraded":"2024-06-18T13:03:07.444674+0000","last_fullsized":"2024-06-18T13:03:07.444674+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:04:48.638790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"69'11","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445182+0000","last_change":"2024-06-18T13:02:32.414932+0000","last_active":"2024-06-18T13:03:07.445182+0000","last_peered":"2024-06-18T13:03:07.445182+0000","last_clean":"2024-06-18T13:03:07.445182+0000","last_became_active":"2024-06-18T13:02:32.414418+0000","last_became_peered":"2024-06-18T13:02:32.414418+0000","last_unstale":"2024-06-18T13:03:07.445182+0000","last_undegraded":"2024-06-18T13:03:07.445182+0000","last_fullsized":"2024-06-18T13:03:07.445182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:34:50.286768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417495+0000","last_change":"2024-06-18T13:02:30.420772+0000","last_active":"2024-06-18T13:03:07.417495+0000","last_peered":"2024-06-18T13:03:07.417495+0000","last_clean":"2024-06-18T13:03:07.417495+0000","last_became_active":"2024-06-18T13:02:30.420481+0000","last_became_peered":"2024-06-18T13:02:30.420481+0000","last_unstale":"2024-06-18T13:03:07.417495+0000","last_undegraded":"2024-06-18T13:03:07.417495+0000","last_fullsized":"2024-06-18T13:03:07.417495+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:06:19.830117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445194+0000","last_change":"2024-06-18T13:02:34.436353+0000","last_active":"2024-06-18T13:03:07.445194+0000","last_peered":"2024-06-18T13:03:07.445194+0000","last_clean":"2024-06-18T13:03:07.445194+0000","last_became_active":"2024-06-18T13:02:34.435995+0000","last_became_peered":"2024-06-18T13:02:34.435995+0000","last_unstale":"2024-06-18T13:03:07.445194+0000","last_undegraded":"2024-06-18T13:03:07.445194+0000","last_fullsized":"2024-06-18T13:03:07.445194+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:39:14.468218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420092+0000","last_change":"2024-06-18T13:02:36.470098+0000","last_active":"2024-06-18T13:03:07.420092+0000","last_peered":"2024-06-18T13:03:07.420092+0000","last_clean":"2024-06-18T13:03:07.420092+0000","last_became_active":"2024-06-18T13:02:36.469759+0000","last_became_peered":"2024-06-18T13:02:36.469759+0000","last_unstale":"2024-06-18T13:03:07.420092+0000","last_undegraded":"2024-06-18T13:03:07.420092+0000","last_fullsized":"2024-06-18T13:03:07.420092+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:27:52.518133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"69'14","reported_seq":46,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443713+0000","last_change":"2024-06-18T13:02:32.414312+0000","last_active":"2024-06-18T13:03:07.443713+0000","last_peered":"2024-06-18T13:03:07.443713+0000","last_clean":"2024-06-18T13:03:07.443713+0000","last_became_active":"2024-06-18T13:02:32.414027+0000","last_became_peered":"2024-06-18T13:02:32.414027+0000","last_unstale":"2024-06-18T13:03:07.443713+0000","last_undegraded":"2024-06-18T13:03:07.443713+0000","last_fullsized":"2024-06-18T13:03:07.443713+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:18:09.361974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'1","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444191+0000","last_change":"2024-06-18T13:02:30.415838+0000","last_active":"2024-06-18T13:03:07.444191+0000","last_peered":"2024-06-18T13:03:07.444191+0000","last_clean":"2024-06-18T13:03:07.444191+0000","last_became_active":"2024-06-18T13:02:30.415576+0000","last_became_peered":"2024-06-18T13:02:30.415576+0000","last_unstale":"2024-06-18T13:03:07.444191+0000","last_undegraded":"2024-06-18T13:03:07.444191+0000","last_fullsized":"2024-06-18T13:03:07.444191+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:57:31.939298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"69'8","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:45.475559+0000","last_change":"2024-06-18T13:02:34.440627+0000","last_active":"2024-06-18T13:03:45.475559+0000","last_peered":"2024-06-18T13:03:45.475559+0000","last_clean":"2024-06-18T13:03:45.475559+0000","last_became_active":"2024-06-18T13:02:34.440273+0000","last_became_peered":"2024-06-18T13:02:34.440273+0000","last_unstale":"2024-06-18T13:03:45.475559+0000","last_undegraded":"2024-06-18T13:03:45.475559+0000","last_fullsized":"2024-06-18T13:03:45.475559+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:14:06.011098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417927+0000","last_change":"2024-06-18T13:02:36.469843+0000","last_active":"2024-06-18T13:03:07.417927+0000","last_peered":"2024-06-18T13:03:07.417927+0000","last_clean":"2024-06-18T13:03:07.417927+0000","last_became_active":"2024-06-18T13:02:36.469587+0000","last_became_peered":"2024-06-18T13:02:36.469587+0000","last_unstale":"2024-06-18T13:03:07.417927+0000","last_undegraded":"2024-06-18T13:03:07.417927+0000","last_fullsized":"2024-06-18T13:03:07.417927+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:52:18.746476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"69'18","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419331+0000","last_change":"2024-06-18T13:02:32.534891+0000","last_active":"2024-06-18T13:03:07.419331+0000","last_peered":"2024-06-18T13:03:07.419331+0000","last_clean":"2024-06-18T13:03:07.419331+0000","last_became_active":"2024-06-18T13:02:32.534560+0000","last_became_peered":"2024-06-18T13:02:32.534560+0000","last_unstale":"2024-06-18T13:03:07.419331+0000","last_undegraded":"2024-06-18T13:03:07.419331+0000","last_fullsized":"2024-06-18T13:03:07.419331+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:46:33.607910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"62'1","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.422810+0000","last_change":"2024-06-18T13:02:30.415028+0000","last_active":"2024-06-18T13:03:07.422810+0000","last_peered":"2024-06-18T13:03:07.422810+0000","last_clean":"2024-06-18T13:03:07.422810+0000","last_became_active":"2024-06-18T13:02:30.414736+0000","last_became_peered":"2024-06-18T13:02:30.414736+0000","last_unstale":"2024-06-18T13:03:07.422810+0000","last_undegraded":"2024-06-18T13:03:07.422810+0000","last_fullsized":"2024-06-18T13:03:07.422810+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:50:27.804864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443486+0000","last_change":"2024-06-18T13:02:34.438695+0000","last_active":"2024-06-18T13:03:07.443486+0000","last_peered":"2024-06-18T13:03:07.443486+0000","last_clean":"2024-06-18T13:03:07.443486+0000","last_became_active":"2024-06-18T13:02:34.438414+0000","last_became_peered":"2024-06-18T13:02:34.438414+0000","last_unstale":"2024-06-18T13:03:07.443486+0000","last_undegraded":"2024-06-18T13:03:07.443486+0000","last_fullsized":"2024-06-18T13:03:07.443486+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:54:28.655422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444440+0000","last_change":"2024-06-18T13:02:36.451124+0000","last_active":"2024-06-18T13:03:07.444440+0000","last_peered":"2024-06-18T13:03:07.444440+0000","last_clean":"2024-06-18T13:03:07.444440+0000","last_became_active":"2024-06-18T13:02:36.450740+0000","last_became_peered":"2024-06-18T13:02:36.450740+0000","last_unstale":"2024-06-18T13:03:07.444440+0000","last_undegraded":"2024-06-18T13:03:07.444440+0000","last_fullsized":"2024-06-18T13:03:07.444440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-20T00:53:37.173230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"69'19","reported_seq":61,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443624+0000","last_change":"2024-06-18T13:02:32.533391+0000","last_active":"2024-06-18T13:03:07.443624+0000","last_peered":"2024-06-18T13:03:07.443624+0000","last_clean":"2024-06-18T13:03:07.443624+0000","last_became_active":"2024-06-18T13:02:32.533062+0000","last_became_peered":"2024-06-18T13:02:32.533062+0000","last_unstale":"2024-06-18T13:03:07.443624+0000","last_undegraded":"2024-06-18T13:03:07.443624+0000","last_fullsized":"2024-06-18T13:03:07.443624+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:53:58.154183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.418568+0000","last_change":"2024-06-18T13:02:30.419269+0000","last_active":"2024-06-18T13:03:07.418568+0000","last_peered":"2024-06-18T13:03:07.418568+0000","last_clean":"2024-06-18T13:03:07.418568+0000","last_became_active":"2024-06-18T13:02:30.418630+0000","last_became_peered":"2024-06-18T13:02:30.418630+0000","last_unstale":"2024-06-18T13:03:07.418568+0000","last_undegraded":"2024-06-18T13:03:07.418568+0000","last_fullsized":"2024-06-18T13:03:07.418568+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-20T00:31:55.971561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443243+0000","last_change":"2024-06-18T13:02:34.444164+0000","last_active":"2024-06-18T13:03:07.443243+0000","last_peered":"2024-06-18T13:03:07.443243+0000","last_clean":"2024-06-18T13:03:07.443243+0000","last_became_active":"2024-06-18T13:02:34.443851+0000","last_became_peered":"2024-06-18T13:02:34.443851+0000","last_unstale":"2024-06-18T13:03:07.443243+0000","last_undegraded":"2024-06-18T13:03:07.443243+0000","last_fullsized":"2024-06-18T13:03:07.443243+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:25:41.611721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420699+0000","last_change":"2024-06-18T13:02:36.469888+0000","last_active":"2024-06-18T13:03:07.420699+0000","last_peered":"2024-06-18T13:03:07.420699+0000","last_clean":"2024-06-18T13:03:07.420699+0000","last_became_active":"2024-06-18T13:02:36.469269+0000","last_became_peered":"2024-06-18T13:02:36.469269+0000","last_unstale":"2024-06-18T13:03:07.420699+0000","last_undegraded":"2024-06-18T13:03:07.420699+0000","last_fullsized":"2024-06-18T13:03:07.420699+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T22:52:31.531988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"69'10","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.423411+0000","last_change":"2024-06-18T13:02:32.412282+0000","last_active":"2024-06-18T13:03:07.423411+0000","last_peered":"2024-06-18T13:03:07.423411+0000","last_clean":"2024-06-18T13:03:07.423411+0000","last_became_active":"2024-06-18T13:02:32.411958+0000","last_became_peered":"2024-06-18T13:02:32.411958+0000","last_unstale":"2024-06-18T13:03:07.423411+0000","last_undegraded":"2024-06-18T13:03:07.423411+0000","last_fullsized":"2024-06-18T13:03:07.423411+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:54:17.816941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.421804+0000","last_change":"2024-06-18T13:02:30.412493+0000","last_active":"2024-06-18T13:03:07.421804+0000","last_peered":"2024-06-18T13:03:07.421804+0000","last_clean":"2024-06-18T13:03:07.421804+0000","last_became_active":"2024-06-18T13:02:30.412192+0000","last_became_peered":"2024-06-18T13:02:30.412192+0000","last_unstale":"2024-06-18T13:03:07.421804+0000","last_undegraded":"2024-06-18T13:03:07.421804+0000","last_fullsized":"2024-06-18T13:03:07.421804+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:04:53.921051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"69'8","reported_seq":46,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:45.475544+0000","last_change":"2024-06-18T13:02:34.438824+0000","last_active":"2024-06-18T13:03:45.475544+0000","last_peered":"2024-06-18T13:03:45.475544+0000","last_clean":"2024-06-18T13:03:45.475544+0000","last_became_active":"2024-06-18T13:02:34.438505+0000","last_became_peered":"2024-06-18T13:02:34.438505+0000","last_unstale":"2024-06-18T13:03:45.475544+0000","last_undegraded":"2024-06-18T13:03:45.475544+0000","last_fullsized":"2024-06-18T13:03:45.475544+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:12:02.840238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443073+0000","last_change":"2024-06-18T13:02:36.452331+0000","last_active":"2024-06-18T13:03:07.443073+0000","last_peered":"2024-06-18T13:03:07.443073+0000","last_clean":"2024-06-18T13:03:07.443073+0000","last_became_active":"2024-06-18T13:02:36.452030+0000","last_became_peered":"2024-06-18T13:02:36.452030+0000","last_unstale":"2024-06-18T13:03:07.443073+0000","last_undegraded":"2024-06-18T13:03:07.443073+0000","last_fullsized":"2024-06-18T13:03:07.443073+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:29:50.986345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"69'28","reported_seq":90,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:45.475640+0000","last_change":"2024-06-18T13:02:32.421366+0000","last_active":"2024-06-18T13:03:45.475640+0000","last_peered":"2024-06-18T13:03:45.475640+0000","last_clean":"2024-06-18T13:03:45.475640+0000","last_became_active":"2024-06-18T13:02:32.421039+0000","last_became_peered":"2024-06-18T13:02:32.421039+0000","last_unstale":"2024-06-18T13:03:45.475640+0000","last_undegraded":"2024-06-18T13:03:45.475640+0000","last_fullsized":"2024-06-18T13:03:45.475640+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:14:02.563774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444232+0000","last_change":"2024-06-18T13:02:30.412383+0000","last_active":"2024-06-18T13:03:07.444232+0000","last_peered":"2024-06-18T13:03:07.444232+0000","last_clean":"2024-06-18T13:03:07.444232+0000","last_became_active":"2024-06-18T13:02:30.412097+0000","last_became_peered":"2024-06-18T13:02:30.412097+0000","last_unstale":"2024-06-18T13:03:07.444232+0000","last_undegraded":"2024-06-18T13:03:07.444232+0000","last_fullsized":"2024-06-18T13:03:07.444232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:48:14.289434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443673+0000","last_change":"2024-06-18T13:02:34.432634+0000","last_active":"2024-06-18T13:03:07.443673+0000","last_peered":"2024-06-18T13:03:07.443673+0000","last_clean":"2024-06-18T13:03:07.443673+0000","last_became_active":"2024-06-18T13:02:34.432341+0000","last_became_peered":"2024-06-18T13:02:34.432341+0000","last_unstale":"2024-06-18T13:03:07.443673+0000","last_undegraded":"2024-06-18T13:03:07.443673+0000","last_fullsized":"2024-06-18T13:03:07.443673+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T22:46:19.795026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420729+0000","last_change":"2024-06-18T13:02:36.451323+0000","last_active":"2024-06-18T13:03:07.420729+0000","last_peered":"2024-06-18T13:03:07.420729+0000","last_clean":"2024-06-18T13:03:07.420729+0000","last_became_active":"2024-06-18T13:02:36.450594+0000","last_became_peered":"2024-06-18T13:02:36.450594+0000","last_unstale":"2024-06-18T13:03:07.420729+0000","last_undegraded":"2024-06-18T13:03:07.420729+0000","last_fullsized":"2024-06-18T13:03:07.420729+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:44:23.737894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"69'13","reported_seq":52,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.423279+0000","last_change":"2024-06-18T13:02:32.417395+0000","last_active":"2024-06-18T13:03:07.423279+0000","last_peered":"2024-06-18T13:03:07.423279+0000","last_clean":"2024-06-18T13:03:07.423279+0000","last_became_active":"2024-06-18T13:02:32.417100+0000","last_became_peered":"2024-06-18T13:02:32.417100+0000","last_unstale":"2024-06-18T13:03:07.423279+0000","last_undegraded":"2024-06-18T13:03:07.423279+0000","last_fullsized":"2024-06-18T13:03:07.423279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:23:10.045343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"69'5","reported_seq":115,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:51.535160+0000","last_change":"2024-06-18T13:02:34.434651+0000","last_active":"2024-06-18T13:03:51.535160+0000","last_peered":"2024-06-18T13:03:51.535160+0000","last_clean":"2024-06-18T13:03:51.535160+0000","last_became_active":"2024-06-18T13:02:30.415191+0000","last_became_peered":"2024-06-18T13:02:30.415191+0000","last_unstale":"2024-06-18T13:03:51.535160+0000","last_undegraded":"2024-06-18T13:03:51.535160+0000","last_fullsized":"2024-06-18T13:03:51.535160+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:45:40.393421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089436150000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443390+0000","last_change":"2024-06-18T13:02:30.413876+0000","last_active":"2024-06-18T13:03:07.443390+0000","last_peered":"2024-06-18T13:03:07.443390+0000","last_clean":"2024-06-18T13:03:07.443390+0000","last_became_active":"2024-06-18T13:02:30.413558+0000","last_became_peered":"2024-06-18T13:02:30.413558+0000","last_unstale":"2024-06-18T13:03:07.443390+0000","last_undegraded":"2024-06-18T13:03:07.443390+0000","last_fullsized":"2024-06-18T13:03:07.443390+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:54:10.596757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443940+0000","last_change":"2024-06-18T13:02:34.435612+0000","last_active":"2024-06-18T13:03:07.443940+0000","last_peered":"2024-06-18T13:03:07.443940+0000","last_clean":"2024-06-18T13:03:07.443940+0000","last_became_active":"2024-06-18T13:02:34.435181+0000","last_became_peered":"2024-06-18T13:02:34.435181+0000","last_unstale":"2024-06-18T13:03:07.443940+0000","last_undegraded":"2024-06-18T13:03:07.443940+0000","last_fullsized":"2024-06-18T13:03:07.443940+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:23:20.217222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.418210+0000","last_change":"2024-06-18T13:02:36.469708+0000","last_active":"2024-06-18T13:03:07.418210+0000","last_peered":"2024-06-18T13:03:07.418210+0000","last_clean":"2024-06-18T13:03:07.418210+0000","last_became_active":"2024-06-18T13:02:36.469428+0000","last_became_peered":"2024-06-18T13:02:36.469428+0000","last_unstale":"2024-06-18T13:03:07.418210+0000","last_undegraded":"2024-06-18T13:03:07.418210+0000","last_fullsized":"2024-06-18T13:03:07.418210+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:46:54.170335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"69'12","reported_seq":43,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444596+0000","last_change":"2024-06-18T13:02:32.419047+0000","last_active":"2024-06-18T13:03:07.444596+0000","last_peered":"2024-06-18T13:03:07.444596+0000","last_clean":"2024-06-18T13:03:07.444596+0000","last_became_active":"2024-06-18T13:02:32.418558+0000","last_became_peered":"2024-06-18T13:02:32.418558+0000","last_unstale":"2024-06-18T13:03:07.444596+0000","last_undegraded":"2024-06-18T13:03:07.444596+0000","last_fullsized":"2024-06-18T13:03:07.444596+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:15:55.070054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":36,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443539+0000","last_change":"2024-06-18T13:02:34.424339+0000","last_active":"2024-06-18T13:03:07.443539+0000","last_peered":"2024-06-18T13:03:07.443539+0000","last_clean":"2024-06-18T13:03:07.443539+0000","last_became_active":"2024-06-18T13:02:30.417572+0000","last_became_peered":"2024-06-18T13:02:30.417572+0000","last_unstale":"2024-06-18T13:03:07.443539+0000","last_undegraded":"2024-06-18T13:03:07.443539+0000","last_fullsized":"2024-06-18T13:03:07.443539+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:36:03.626819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050054199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417600+0000","last_change":"2024-06-18T13:02:30.420941+0000","last_active":"2024-06-18T13:03:07.417600+0000","last_peered":"2024-06-18T13:03:07.417600+0000","last_clean":"2024-06-18T13:03:07.417600+0000","last_became_active":"2024-06-18T13:02:30.420700+0000","last_became_peered":"2024-06-18T13:02:30.420700+0000","last_unstale":"2024-06-18T13:03:07.417600+0000","last_undegraded":"2024-06-18T13:03:07.417600+0000","last_fullsized":"2024-06-18T13:03:07.417600+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:28:06.339022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445195+0000","last_change":"2024-06-18T13:02:34.439543+0000","last_active":"2024-06-18T13:03:07.445195+0000","last_peered":"2024-06-18T13:03:07.445195+0000","last_clean":"2024-06-18T13:03:07.445195+0000","last_became_active":"2024-06-18T13:02:34.438871+0000","last_became_peered":"2024-06-18T13:02:34.438871+0000","last_unstale":"2024-06-18T13:03:07.445195+0000","last_undegraded":"2024-06-18T13:03:07.445195+0000","last_fullsized":"2024-06-18T13:03:07.445195+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:18:39.026087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.421380+0000","last_change":"2024-06-18T13:02:36.451108+0000","last_active":"2024-06-18T13:03:07.421380+0000","last_peered":"2024-06-18T13:03:07.421380+0000","last_clean":"2024-06-18T13:03:07.421380+0000","last_became_active":"2024-06-18T13:02:36.450771+0000","last_became_peered":"2024-06-18T13:02:36.450771+0000","last_unstale":"2024-06-18T13:03:07.421380+0000","last_undegraded":"2024-06-18T13:03:07.421380+0000","last_fullsized":"2024-06-18T13:03:07.421380+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:57:16.812917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"69'16","reported_seq":64,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:45.475927+0000","last_change":"2024-06-18T13:02:32.533948+0000","last_active":"2024-06-18T13:03:45.475927+0000","last_peered":"2024-06-18T13:03:45.475927+0000","last_clean":"2024-06-18T13:03:45.475927+0000","last_became_active":"2024-06-18T13:02:32.533648+0000","last_became_peered":"2024-06-18T13:02:32.533648+0000","last_unstale":"2024-06-18T13:03:45.475927+0000","last_undegraded":"2024-06-18T13:03:45.475927+0000","last_fullsized":"2024-06-18T13:03:45.475927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:20:47.152859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"66'2","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419602+0000","last_change":"2024-06-18T13:02:34.434675+0000","last_active":"2024-06-18T13:03:07.419602+0000","last_peered":"2024-06-18T13:03:07.419602+0000","last_clean":"2024-06-18T13:03:07.419602+0000","last_became_active":"2024-06-18T13:02:30.418683+0000","last_became_peered":"2024-06-18T13:02:30.418683+0000","last_unstale":"2024-06-18T13:03:07.419602+0000","last_undegraded":"2024-06-18T13:03:07.419602+0000","last_fullsized":"2024-06-18T13:03:07.419602+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:23:17.212638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027193109999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444493+0000","last_change":"2024-06-18T13:02:30.413336+0000","last_active":"2024-06-18T13:03:07.444493+0000","last_peered":"2024-06-18T13:03:07.444493+0000","last_clean":"2024-06-18T13:03:07.444493+0000","last_became_active":"2024-06-18T13:02:30.412973+0000","last_became_peered":"2024-06-18T13:02:30.412973+0000","last_unstale":"2024-06-18T13:03:07.444493+0000","last_undegraded":"2024-06-18T13:03:07.444493+0000","last_fullsized":"2024-06-18T13:03:07.444493+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:54:27.312192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":275,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:09.499371+0000","last_change":"2024-06-18T13:02:18.639016+0000","last_active":"2024-06-18T13:03:09.499371+0000","last_peered":"2024-06-18T13:03:09.499371+0000","last_clean":"2024-06-18T13:03:09.499371+0000","last_became_active":"2024-06-18T13:02:18.638579+0000","last_became_peered":"2024-06-18T13:02:18.638579+0000","last_unstale":"2024-06-18T13:03:09.499371+0000","last_undegraded":"2024-06-18T13:03:09.499371+0000","last_fullsized":"2024-06-18T13:03:09.499371+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:00:22.175291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:00:22.175291+0000","last_clean_scrub_stamp":"2024-06-18T13:00:22.175291+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:24:19.503964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.418033+0000","last_change":"2024-06-18T13:02:34.442649+0000","last_active":"2024-06-18T13:03:07.418033+0000","last_peered":"2024-06-18T13:03:07.418033+0000","last_clean":"2024-06-18T13:03:07.418033+0000","last_became_active":"2024-06-18T13:02:34.442349+0000","last_became_peered":"2024-06-18T13:02:34.442349+0000","last_unstale":"2024-06-18T13:03:07.418033+0000","last_undegraded":"2024-06-18T13:03:07.418033+0000","last_fullsized":"2024-06-18T13:03:07.418033+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:14:17.239342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445250+0000","last_change":"2024-06-18T13:02:36.446367+0000","last_active":"2024-06-18T13:03:07.445250+0000","last_peered":"2024-06-18T13:03:07.445250+0000","last_clean":"2024-06-18T13:03:07.445250+0000","last_became_active":"2024-06-18T13:02:36.446060+0000","last_became_peered":"2024-06-18T13:02:36.446060+0000","last_unstale":"2024-06-18T13:03:07.445250+0000","last_undegraded":"2024-06-18T13:03:07.445250+0000","last_fullsized":"2024-06-18T13:03:07.445250+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T22:10:50.569169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"69'19","reported_seq":56,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443414+0000","last_change":"2024-06-18T13:02:32.533548+0000","last_active":"2024-06-18T13:03:07.443414+0000","last_peered":"2024-06-18T13:03:07.443414+0000","last_clean":"2024-06-18T13:03:07.443414+0000","last_became_active":"2024-06-18T13:02:32.533224+0000","last_became_peered":"2024-06-18T13:02:32.533224+0000","last_unstale":"2024-06-18T13:03:07.443414+0000","last_undegraded":"2024-06-18T13:03:07.443414+0000","last_fullsized":"2024-06-18T13:03:07.443414+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:30:29.678869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443915+0000","last_change":"2024-06-18T13:02:30.421233+0000","last_active":"2024-06-18T13:03:07.443915+0000","last_peered":"2024-06-18T13:03:07.443915+0000","last_clean":"2024-06-18T13:03:07.443915+0000","last_became_active":"2024-06-18T13:02:30.420935+0000","last_became_peered":"2024-06-18T13:02:30.420935+0000","last_unstale":"2024-06-18T13:03:07.443915+0000","last_undegraded":"2024-06-18T13:03:07.443915+0000","last_fullsized":"2024-06-18T13:03:07.443915+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:47:11.104838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443464+0000","last_change":"2024-06-18T13:02:34.435847+0000","last_active":"2024-06-18T13:03:07.443464+0000","last_peered":"2024-06-18T13:03:07.443464+0000","last_clean":"2024-06-18T13:03:07.443464+0000","last_became_active":"2024-06-18T13:02:34.435545+0000","last_became_peered":"2024-06-18T13:02:34.435545+0000","last_unstale":"2024-06-18T13:03:07.443464+0000","last_undegraded":"2024-06-18T13:03:07.443464+0000","last_fullsized":"2024-06-18T13:03:07.443464+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:22:04.151349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"69'1","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417680+0000","last_change":"2024-06-18T13:02:36.451501+0000","last_active":"2024-06-18T13:03:07.417680+0000","last_peered":"2024-06-18T13:03:07.417680+0000","last_clean":"2024-06-18T13:03:07.417680+0000","last_became_active":"2024-06-18T13:02:36.451215+0000","last_became_peered":"2024-06-18T13:02:36.451215+0000","last_unstale":"2024-06-18T13:03:07.417680+0000","last_undegraded":"2024-06-18T13:03:07.417680+0000","last_fullsized":"2024-06-18T13:03:07.417680+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T20:21:42.922203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"69'9","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443499+0000","last_change":"2024-06-18T13:02:32.419605+0000","last_active":"2024-06-18T13:03:07.443499+0000","last_peered":"2024-06-18T13:03:07.443499+0000","last_clean":"2024-06-18T13:03:07.443499+0000","last_became_active":"2024-06-18T13:02:32.419275+0000","last_became_peered":"2024-06-18T13:02:32.419275+0000","last_unstale":"2024-06-18T13:03:07.443499+0000","last_undegraded":"2024-06-18T13:03:07.443499+0000","last_fullsized":"2024-06-18T13:03:07.443499+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:21:14.031837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.422462+0000","last_change":"2024-06-18T13:02:30.414122+0000","last_active":"2024-06-18T13:03:07.422462+0000","last_peered":"2024-06-18T13:03:07.422462+0000","last_clean":"2024-06-18T13:03:07.422462+0000","last_became_active":"2024-06-18T13:02:30.413848+0000","last_became_peered":"2024-06-18T13:02:30.413848+0000","last_unstale":"2024-06-18T13:03:07.422462+0000","last_undegraded":"2024-06-18T13:03:07.422462+0000","last_fullsized":"2024-06-18T13:03:07.422462+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:39:51.884607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443607+0000","last_change":"2024-06-18T13:02:34.435744+0000","last_active":"2024-06-18T13:03:07.443607+0000","last_peered":"2024-06-18T13:03:07.443607+0000","last_clean":"2024-06-18T13:03:07.443607+0000","last_became_active":"2024-06-18T13:02:34.435441+0000","last_became_peered":"2024-06-18T13:02:34.435441+0000","last_unstale":"2024-06-18T13:03:07.443607+0000","last_undegraded":"2024-06-18T13:03:07.443607+0000","last_fullsized":"2024-06-18T13:03:07.443607+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:38:46.429243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443448+0000","last_change":"2024-06-18T13:02:36.448885+0000","last_active":"2024-06-18T13:03:07.443448+0000","last_peered":"2024-06-18T13:03:07.443448+0000","last_clean":"2024-06-18T13:03:07.443448+0000","last_became_active":"2024-06-18T13:02:36.448529+0000","last_became_peered":"2024-06-18T13:02:36.448529+0000","last_unstale":"2024-06-18T13:03:07.443448+0000","last_undegraded":"2024-06-18T13:03:07.443448+0000","last_fullsized":"2024-06-18T13:03:07.443448+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:01:21.884620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"69'15","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443227+0000","last_change":"2024-06-18T13:02:32.534570+0000","last_active":"2024-06-18T13:03:07.443227+0000","last_peered":"2024-06-18T13:03:07.443227+0000","last_clean":"2024-06-18T13:03:07.443227+0000","last_became_active":"2024-06-18T13:02:32.534103+0000","last_became_peered":"2024-06-18T13:02:32.534103+0000","last_unstale":"2024-06-18T13:03:07.443227+0000","last_undegraded":"2024-06-18T13:03:07.443227+0000","last_fullsized":"2024-06-18T13:03:07.443227+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:50:17.718171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444082+0000","last_change":"2024-06-18T13:02:30.417488+0000","last_active":"2024-06-18T13:03:07.444082+0000","last_peered":"2024-06-18T13:03:07.444082+0000","last_clean":"2024-06-18T13:03:07.444082+0000","last_became_active":"2024-06-18T13:02:30.416850+0000","last_became_peered":"2024-06-18T13:02:30.416850+0000","last_unstale":"2024-06-18T13:03:07.444082+0000","last_undegraded":"2024-06-18T13:03:07.444082+0000","last_fullsized":"2024-06-18T13:03:07.444082+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:00:24.082688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"69'8","reported_seq":46,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:45.475623+0000","last_change":"2024-06-18T13:02:34.437602+0000","last_active":"2024-06-18T13:03:45.475623+0000","last_peered":"2024-06-18T13:03:45.475623+0000","last_clean":"2024-06-18T13:03:45.475623+0000","last_became_active":"2024-06-18T13:02:34.437067+0000","last_became_peered":"2024-06-18T13:02:34.437067+0000","last_unstale":"2024-06-18T13:03:45.475623+0000","last_undegraded":"2024-06-18T13:03:45.475623+0000","last_fullsized":"2024-06-18T13:03:45.475623+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:53:26.262004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443859+0000","last_change":"2024-06-18T13:02:36.469535+0000","last_active":"2024-06-18T13:03:07.443859+0000","last_peered":"2024-06-18T13:03:07.443859+0000","last_clean":"2024-06-18T13:03:07.443859+0000","last_became_active":"2024-06-18T13:02:36.469236+0000","last_became_peered":"2024-06-18T13:02:36.469236+0000","last_unstale":"2024-06-18T13:03:07.443859+0000","last_undegraded":"2024-06-18T13:03:07.443859+0000","last_fullsized":"2024-06-18T13:03:07.443859+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:13:11.671125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"69'12","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443579+0000","last_change":"2024-06-18T13:02:32.418020+0000","last_active":"2024-06-18T13:03:07.443579+0000","last_peered":"2024-06-18T13:03:07.443579+0000","last_clean":"2024-06-18T13:03:07.443579+0000","last_became_active":"2024-06-18T13:02:32.417586+0000","last_became_peered":"2024-06-18T13:02:32.417586+0000","last_unstale":"2024-06-18T13:03:07.443579+0000","last_undegraded":"2024-06-18T13:03:07.443579+0000","last_fullsized":"2024-06-18T13:03:07.443579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:24:49.047854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"62'2","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445325+0000","last_change":"2024-06-18T13:02:30.415409+0000","last_active":"2024-06-18T13:03:07.445325+0000","last_peered":"2024-06-18T13:03:07.445325+0000","last_clean":"2024-06-18T13:03:07.445325+0000","last_became_active":"2024-06-18T13:02:30.415071+0000","last_became_peered":"2024-06-18T13:02:30.415071+0000","last_unstale":"2024-06-18T13:03:07.445325+0000","last_undegraded":"2024-06-18T13:03:07.445325+0000","last_fullsized":"2024-06-18T13:03:07.445325+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:51:42.249337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443084+0000","last_change":"2024-06-18T13:02:34.433724+0000","last_active":"2024-06-18T13:03:07.443084+0000","last_peered":"2024-06-18T13:03:07.443084+0000","last_clean":"2024-06-18T13:03:07.443084+0000","last_became_active":"2024-06-18T13:02:34.433352+0000","last_became_peered":"2024-06-18T13:02:34.433352+0000","last_unstale":"2024-06-18T13:03:07.443084+0000","last_undegraded":"2024-06-18T13:03:07.443084+0000","last_fullsized":"2024-06-18T13:03:07.443084+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:34:38.238250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419707+0000","last_change":"2024-06-18T13:02:36.452292+0000","last_active":"2024-06-18T13:03:07.419707+0000","last_peered":"2024-06-18T13:03:07.419707+0000","last_clean":"2024-06-18T13:03:07.419707+0000","last_became_active":"2024-06-18T13:02:36.450576+0000","last_became_peered":"2024-06-18T13:02:36.450576+0000","last_unstale":"2024-06-18T13:03:07.419707+0000","last_undegraded":"2024-06-18T13:03:07.419707+0000","last_fullsized":"2024-06-18T13:03:07.419707+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:50:24.609427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"69'9","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444532+0000","last_change":"2024-06-18T13:02:32.533559+0000","last_active":"2024-06-18T13:03:07.444532+0000","last_peered":"2024-06-18T13:03:07.444532+0000","last_clean":"2024-06-18T13:03:07.444532+0000","last_became_active":"2024-06-18T13:02:32.533271+0000","last_became_peered":"2024-06-18T13:02:32.533271+0000","last_unstale":"2024-06-18T13:03:07.444532+0000","last_undegraded":"2024-06-18T13:03:07.444532+0000","last_fullsized":"2024-06-18T13:03:07.444532+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:07:55.627893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443090+0000","last_change":"2024-06-18T13:02:30.412180+0000","last_active":"2024-06-18T13:03:07.443090+0000","last_peered":"2024-06-18T13:03:07.443090+0000","last_clean":"2024-06-18T13:03:07.443090+0000","last_became_active":"2024-06-18T13:02:30.411887+0000","last_became_peered":"2024-06-18T13:02:30.411887+0000","last_unstale":"2024-06-18T13:03:07.443090+0000","last_undegraded":"2024-06-18T13:03:07.443090+0000","last_fullsized":"2024-06-18T13:03:07.443090+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:47:52.143479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420472+0000","last_change":"2024-06-18T13:02:34.446713+0000","last_active":"2024-06-18T13:03:07.420472+0000","last_peered":"2024-06-18T13:03:07.420472+0000","last_clean":"2024-06-18T13:03:07.420472+0000","last_became_active":"2024-06-18T13:02:34.446455+0000","last_became_peered":"2024-06-18T13:02:34.446455+0000","last_unstale":"2024-06-18T13:03:07.420472+0000","last_undegraded":"2024-06-18T13:03:07.420472+0000","last_fullsized":"2024-06-18T13:03:07.420472+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:47:23.223424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443409+0000","last_change":"2024-06-18T13:02:36.444328+0000","last_active":"2024-06-18T13:03:07.443409+0000","last_peered":"2024-06-18T13:03:07.443409+0000","last_clean":"2024-06-18T13:03:07.443409+0000","last_became_active":"2024-06-18T13:02:36.443639+0000","last_became_peered":"2024-06-18T13:02:36.443639+0000","last_unstale":"2024-06-18T13:03:07.443409+0000","last_undegraded":"2024-06-18T13:03:07.443409+0000","last_fullsized":"2024-06-18T13:03:07.443409+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:58:08.219150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443728+0000","last_change":"2024-06-18T13:02:30.416751+0000","last_active":"2024-06-18T13:03:07.443728+0000","last_peered":"2024-06-18T13:03:07.443728+0000","last_clean":"2024-06-18T13:03:07.443728+0000","last_became_active":"2024-06-18T13:02:30.416493+0000","last_became_peered":"2024-06-18T13:02:30.416493+0000","last_unstale":"2024-06-18T13:03:07.443728+0000","last_undegraded":"2024-06-18T13:03:07.443728+0000","last_fullsized":"2024-06-18T13:03:07.443728+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:05:33.797355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"69'6","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420560+0000","last_change":"2024-06-18T13:02:32.534357+0000","last_active":"2024-06-18T13:03:07.420560+0000","last_peered":"2024-06-18T13:03:07.420560+0000","last_clean":"2024-06-18T13:03:07.420560+0000","last_became_active":"2024-06-18T13:02:32.534070+0000","last_became_peered":"2024-06-18T13:02:32.534070+0000","last_unstale":"2024-06-18T13:03:07.420560+0000","last_undegraded":"2024-06-18T13:03:07.420560+0000","last_fullsized":"2024-06-18T13:03:07.420560+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:42:53.361894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444965+0000","last_change":"2024-06-18T13:02:34.441909+0000","last_active":"2024-06-18T13:03:07.444965+0000","last_peered":"2024-06-18T13:03:07.444965+0000","last_clean":"2024-06-18T13:03:07.444965+0000","last_became_active":"2024-06-18T13:02:34.441543+0000","last_became_peered":"2024-06-18T13:02:34.441543+0000","last_unstale":"2024-06-18T13:03:07.444965+0000","last_undegraded":"2024-06-18T13:03:07.444965+0000","last_fullsized":"2024-06-18T13:03:07.444965+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T22:10:20.863847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.418121+0000","last_change":"2024-06-18T13:02:36.471949+0000","last_active":"2024-06-18T13:03:07.418121+0000","last_peered":"2024-06-18T13:03:07.418121+0000","last_clean":"2024-06-18T13:03:07.418121+0000","last_became_active":"2024-06-18T13:02:36.471644+0000","last_became_peered":"2024-06-18T13:02:36.471644+0000","last_unstale":"2024-06-18T13:03:07.418121+0000","last_undegraded":"2024-06-18T13:03:07.418121+0000","last_fullsized":"2024-06-18T13:03:07.418121+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:15:22.042120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"69'10","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420328+0000","last_change":"2024-06-18T13:02:32.533884+0000","last_active":"2024-06-18T13:03:07.420328+0000","last_peered":"2024-06-18T13:03:07.420328+0000","last_clean":"2024-06-18T13:03:07.420328+0000","last_became_active":"2024-06-18T13:02:32.533545+0000","last_became_peered":"2024-06-18T13:02:32.533545+0000","last_unstale":"2024-06-18T13:03:07.420328+0000","last_undegraded":"2024-06-18T13:03:07.420328+0000","last_fullsized":"2024-06-18T13:03:07.420328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:04:33.758753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444196+0000","last_change":"2024-06-18T13:02:30.414245+0000","last_active":"2024-06-18T13:03:07.444196+0000","last_peered":"2024-06-18T13:03:07.444196+0000","last_clean":"2024-06-18T13:03:07.444196+0000","last_became_active":"2024-06-18T13:02:30.413920+0000","last_became_peered":"2024-06-18T13:02:30.413920+0000","last_unstale":"2024-06-18T13:03:07.444196+0000","last_undegraded":"2024-06-18T13:03:07.444196+0000","last_fullsized":"2024-06-18T13:03:07.444196+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:51:02.772980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"69'8","reported_seq":46,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:45.475646+0000","last_change":"2024-06-18T13:02:34.440909+0000","last_active":"2024-06-18T13:03:45.475646+0000","last_peered":"2024-06-18T13:03:45.475646+0000","last_clean":"2024-06-18T13:03:45.475646+0000","last_became_active":"2024-06-18T13:02:34.440538+0000","last_became_peered":"2024-06-18T13:02:34.440538+0000","last_unstale":"2024-06-18T13:03:45.475646+0000","last_undegraded":"2024-06-18T13:03:45.475646+0000","last_fullsized":"2024-06-18T13:03:45.475646+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:12:34.973776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444300+0000","last_change":"2024-06-18T13:02:36.454638+0000","last_active":"2024-06-18T13:03:07.444300+0000","last_peered":"2024-06-18T13:03:07.444300+0000","last_clean":"2024-06-18T13:03:07.444300+0000","last_became_active":"2024-06-18T13:02:36.454294+0000","last_became_peered":"2024-06-18T13:02:36.454294+0000","last_unstale":"2024-06-18T13:03:07.444300+0000","last_undegraded":"2024-06-18T13:03:07.444300+0000","last_fullsized":"2024-06-18T13:03:07.444300+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:05:25.333139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"69'9","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444981+0000","last_change":"2024-06-18T13:02:32.533679+0000","last_active":"2024-06-18T13:03:07.444981+0000","last_peered":"2024-06-18T13:03:07.444981+0000","last_clean":"2024-06-18T13:03:07.444981+0000","last_became_active":"2024-06-18T13:02:32.533358+0000","last_became_peered":"2024-06-18T13:02:32.533358+0000","last_unstale":"2024-06-18T13:03:07.444981+0000","last_undegraded":"2024-06-18T13:03:07.444981+0000","last_fullsized":"2024-06-18T13:03:07.444981+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:03:07.091045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443819+0000","last_change":"2024-06-18T13:02:30.415519+0000","last_active":"2024-06-18T13:03:07.443819+0000","last_peered":"2024-06-18T13:03:07.443819+0000","last_clean":"2024-06-18T13:03:07.443819+0000","last_became_active":"2024-06-18T13:02:30.414709+0000","last_became_peered":"2024-06-18T13:02:30.414709+0000","last_unstale":"2024-06-18T13:03:07.443819+0000","last_undegraded":"2024-06-18T13:03:07.443819+0000","last_fullsized":"2024-06-18T13:03:07.443819+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:32:28.153975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"69'8","reported_seq":46,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:45.475866+0000","last_change":"2024-06-18T13:02:34.447600+0000","last_active":"2024-06-18T13:03:45.475866+0000","last_peered":"2024-06-18T13:03:45.475866+0000","last_clean":"2024-06-18T13:03:45.475866+0000","last_became_active":"2024-06-18T13:02:34.447251+0000","last_became_peered":"2024-06-18T13:02:34.447251+0000","last_unstale":"2024-06-18T13:03:45.475866+0000","last_undegraded":"2024-06-18T13:03:45.475866+0000","last_fullsized":"2024-06-18T13:03:45.475866+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:54:11.312071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445084+0000","last_change":"2024-06-18T13:02:36.445621+0000","last_active":"2024-06-18T13:03:07.445084+0000","last_peered":"2024-06-18T13:03:07.445084+0000","last_clean":"2024-06-18T13:03:07.445084+0000","last_became_active":"2024-06-18T13:02:36.445281+0000","last_became_peered":"2024-06-18T13:02:36.445281+0000","last_unstale":"2024-06-18T13:03:07.445084+0000","last_undegraded":"2024-06-18T13:03:07.445084+0000","last_fullsized":"2024-06-18T13:03:07.445084+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:35:37.750819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"69'9","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.423371+0000","last_change":"2024-06-18T13:02:32.415035+0000","last_active":"2024-06-18T13:03:07.423371+0000","last_peered":"2024-06-18T13:03:07.423371+0000","last_clean":"2024-06-18T13:03:07.423371+0000","last_became_active":"2024-06-18T13:02:32.414539+0000","last_became_peered":"2024-06-18T13:02:32.414539+0000","last_unstale":"2024-06-18T13:03:07.423371+0000","last_undegraded":"2024-06-18T13:03:07.423371+0000","last_fullsized":"2024-06-18T13:03:07.423371+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:11:20.956605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444183+0000","last_change":"2024-06-18T13:02:30.414277+0000","last_active":"2024-06-18T13:03:07.444183+0000","last_peered":"2024-06-18T13:03:07.444183+0000","last_clean":"2024-06-18T13:03:07.444183+0000","last_became_active":"2024-06-18T13:02:30.414007+0000","last_became_peered":"2024-06-18T13:02:30.414007+0000","last_unstale":"2024-06-18T13:03:07.444183+0000","last_undegraded":"2024-06-18T13:03:07.444183+0000","last_fullsized":"2024-06-18T13:03:07.444183+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:27:49.237415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419798+0000","last_change":"2024-06-18T13:02:34.438273+0000","last_active":"2024-06-18T13:03:07.419798+0000","last_peered":"2024-06-18T13:03:07.419798+0000","last_clean":"2024-06-18T13:03:07.419798+0000","last_became_active":"2024-06-18T13:02:34.437964+0000","last_became_peered":"2024-06-18T13:02:34.437964+0000","last_unstale":"2024-06-18T13:03:07.419798+0000","last_undegraded":"2024-06-18T13:03:07.419798+0000","last_fullsized":"2024-06-18T13:03:07.419798+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:59:42.364269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443051+0000","last_change":"2024-06-18T13:02:36.453525+0000","last_active":"2024-06-18T13:03:07.443051+0000","last_peered":"2024-06-18T13:03:07.443051+0000","last_clean":"2024-06-18T13:03:07.443051+0000","last_became_active":"2024-06-18T13:02:36.453239+0000","last_became_peered":"2024-06-18T13:02:36.453239+0000","last_unstale":"2024-06-18T13:03:07.443051+0000","last_undegraded":"2024-06-18T13:03:07.443051+0000","last_fullsized":"2024-06-18T13:03:07.443051+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T19:35:26.563010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"69'11","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444608+0000","last_change":"2024-06-18T13:02:32.414219+0000","last_active":"2024-06-18T13:03:07.444608+0000","last_peered":"2024-06-18T13:03:07.444608+0000","last_clean":"2024-06-18T13:03:07.444608+0000","last_became_active":"2024-06-18T13:02:32.413885+0000","last_became_peered":"2024-06-18T13:02:32.413885+0000","last_unstale":"2024-06-18T13:03:07.444608+0000","last_undegraded":"2024-06-18T13:03:07.444608+0000","last_fullsized":"2024-06-18T13:03:07.444608+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-20T00:51:00.963524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.422385+0000","last_change":"2024-06-18T13:02:30.407418+0000","last_active":"2024-06-18T13:03:07.422385+0000","last_peered":"2024-06-18T13:03:07.422385+0000","last_clean":"2024-06-18T13:03:07.422385+0000","last_became_active":"2024-06-18T13:02:30.406648+0000","last_became_peered":"2024-06-18T13:02:30.406648+0000","last_unstale":"2024-06-18T13:03:07.422385+0000","last_undegraded":"2024-06-18T13:03:07.422385+0000","last_fullsized":"2024-06-18T13:03:07.422385+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:16:32.062110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.423228+0000","last_change":"2024-06-18T13:02:34.443388+0000","last_active":"2024-06-18T13:03:07.423228+0000","last_peered":"2024-06-18T13:03:07.423228+0000","last_clean":"2024-06-18T13:03:07.423228+0000","last_became_active":"2024-06-18T13:02:34.442490+0000","last_became_peered":"2024-06-18T13:02:34.442490+0000","last_unstale":"2024-06-18T13:03:07.423228+0000","last_undegraded":"2024-06-18T13:03:07.423228+0000","last_fullsized":"2024-06-18T13:03:07.423228+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:19:17.974739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419207+0000","last_change":"2024-06-18T13:02:36.453644+0000","last_active":"2024-06-18T13:03:07.419207+0000","last_peered":"2024-06-18T13:03:07.419207+0000","last_clean":"2024-06-18T13:03:07.419207+0000","last_became_active":"2024-06-18T13:02:36.453301+0000","last_became_peered":"2024-06-18T13:02:36.453301+0000","last_unstale":"2024-06-18T13:03:07.419207+0000","last_undegraded":"2024-06-18T13:03:07.419207+0000","last_fullsized":"2024-06-18T13:03:07.419207+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:20:49.347688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"69'4","reported_seq":31,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419236+0000","last_change":"2024-06-18T13:02:32.422818+0000","last_active":"2024-06-18T13:03:07.419236+0000","last_peered":"2024-06-18T13:03:07.419236+0000","last_clean":"2024-06-18T13:03:07.419236+0000","last_became_active":"2024-06-18T13:02:32.422429+0000","last_became_peered":"2024-06-18T13:02:32.422429+0000","last_unstale":"2024-06-18T13:03:07.419236+0000","last_undegraded":"2024-06-18T13:03:07.419236+0000","last_fullsized":"2024-06-18T13:03:07.419236+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:59:22.912024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"62'1","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.445056+0000","last_change":"2024-06-18T13:02:30.414551+0000","last_active":"2024-06-18T13:03:07.445056+0000","last_peered":"2024-06-18T13:03:07.445056+0000","last_clean":"2024-06-18T13:03:07.445056+0000","last_became_active":"2024-06-18T13:02:30.414253+0000","last_became_peered":"2024-06-18T13:02:30.414253+0000","last_unstale":"2024-06-18T13:03:07.445056+0000","last_undegraded":"2024-06-18T13:03:07.445056+0000","last_fullsized":"2024-06-18T13:03:07.445056+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:37:38.599058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443549+0000","last_change":"2024-06-18T13:02:34.442532+0000","last_active":"2024-06-18T13:03:07.443549+0000","last_peered":"2024-06-18T13:03:07.443549+0000","last_clean":"2024-06-18T13:03:07.443549+0000","last_became_active":"2024-06-18T13:02:34.442084+0000","last_became_peered":"2024-06-18T13:02:34.442084+0000","last_unstale":"2024-06-18T13:03:07.443549+0000","last_undegraded":"2024-06-18T13:03:07.443549+0000","last_fullsized":"2024-06-18T13:03:07.443549+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-20T00:19:31.477775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417486+0000","last_change":"2024-06-18T13:02:36.448715+0000","last_active":"2024-06-18T13:03:07.417486+0000","last_peered":"2024-06-18T13:03:07.417486+0000","last_clean":"2024-06-18T13:03:07.417486+0000","last_became_active":"2024-06-18T13:02:36.448371+0000","last_became_peered":"2024-06-18T13:02:36.448371+0000","last_unstale":"2024-06-18T13:03:07.417486+0000","last_undegraded":"2024-06-18T13:03:07.417486+0000","last_fullsized":"2024-06-18T13:03:07.417486+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:30:18.503424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"69'11","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419565+0000","last_change":"2024-06-18T13:02:32.534315+0000","last_active":"2024-06-18T13:03:07.419565+0000","last_peered":"2024-06-18T13:03:07.419565+0000","last_clean":"2024-06-18T13:03:07.419565+0000","last_became_active":"2024-06-18T13:02:32.533868+0000","last_became_peered":"2024-06-18T13:02:32.533868+0000","last_unstale":"2024-06-18T13:03:07.419565+0000","last_undegraded":"2024-06-18T13:03:07.419565+0000","last_fullsized":"2024-06-18T13:03:07.419565+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T22:44:00.755969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444114+0000","last_change":"2024-06-18T13:02:30.412351+0000","last_active":"2024-06-18T13:03:07.444114+0000","last_peered":"2024-06-18T13:03:07.444114+0000","last_clean":"2024-06-18T13:03:07.444114+0000","last_became_active":"2024-06-18T13:02:30.412067+0000","last_became_peered":"2024-06-18T13:02:30.412067+0000","last_unstale":"2024-06-18T13:03:07.444114+0000","last_undegraded":"2024-06-18T13:03:07.444114+0000","last_fullsized":"2024-06-18T13:03:07.444114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:45:04.636876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417301+0000","last_change":"2024-06-18T13:02:34.445850+0000","last_active":"2024-06-18T13:03:07.417301+0000","last_peered":"2024-06-18T13:03:07.417301+0000","last_clean":"2024-06-18T13:03:07.417301+0000","last_became_active":"2024-06-18T13:02:34.445524+0000","last_became_peered":"2024-06-18T13:02:34.445524+0000","last_unstale":"2024-06-18T13:03:07.417301+0000","last_undegraded":"2024-06-18T13:03:07.417301+0000","last_fullsized":"2024-06-18T13:03:07.417301+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T14:26:06.694880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419612+0000","last_change":"2024-06-18T13:02:36.471298+0000","last_active":"2024-06-18T13:03:07.419612+0000","last_peered":"2024-06-18T13:03:07.419612+0000","last_clean":"2024-06-18T13:03:07.419612+0000","last_became_active":"2024-06-18T13:02:36.470978+0000","last_became_peered":"2024-06-18T13:02:36.470978+0000","last_unstale":"2024-06-18T13:03:07.419612+0000","last_undegraded":"2024-06-18T13:03:07.419612+0000","last_fullsized":"2024-06-18T13:03:07.419612+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:23:17.370819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"69'1","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.417846+0000","last_change":"2024-06-18T13:02:36.448818+0000","last_active":"2024-06-18T13:03:07.417846+0000","last_peered":"2024-06-18T13:03:07.417846+0000","last_clean":"2024-06-18T13:03:07.417846+0000","last_became_active":"2024-06-18T13:02:36.448532+0000","last_became_peered":"2024-06-18T13:02:36.448532+0000","last_unstale":"2024-06-18T13:03:07.417846+0000","last_undegraded":"2024-06-18T13:03:07.417846+0000","last_fullsized":"2024-06-18T13:03:07.417846+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:52:07.803432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443385+0000","last_change":"2024-06-18T13:02:30.411673+0000","last_active":"2024-06-18T13:03:07.443385+0000","last_peered":"2024-06-18T13:03:07.443385+0000","last_clean":"2024-06-18T13:03:07.443385+0000","last_became_active":"2024-06-18T13:02:30.411369+0000","last_became_peered":"2024-06-18T13:02:30.411369+0000","last_unstale":"2024-06-18T13:03:07.443385+0000","last_undegraded":"2024-06-18T13:03:07.443385+0000","last_fullsized":"2024-06-18T13:03:07.443385+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:17:58.513431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"69'10","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444516+0000","last_change":"2024-06-18T13:02:32.533612+0000","last_active":"2024-06-18T13:03:07.444516+0000","last_peered":"2024-06-18T13:03:07.444516+0000","last_clean":"2024-06-18T13:03:07.444516+0000","last_became_active":"2024-06-18T13:02:32.533351+0000","last_became_peered":"2024-06-18T13:02:32.533351+0000","last_unstale":"2024-06-18T13:03:07.444516+0000","last_undegraded":"2024-06-18T13:03:07.444516+0000","last_fullsized":"2024-06-18T13:03:07.444516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T23:19:26.903661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"69'8","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:45.475637+0000","last_change":"2024-06-18T13:02:34.443478+0000","last_active":"2024-06-18T13:03:45.475637+0000","last_peered":"2024-06-18T13:03:45.475637+0000","last_clean":"2024-06-18T13:03:45.475637+0000","last_became_active":"2024-06-18T13:02:34.442263+0000","last_became_peered":"2024-06-18T13:02:34.442263+0000","last_unstale":"2024-06-18T13:03:45.475637+0000","last_undegraded":"2024-06-18T13:03:45.475637+0000","last_fullsized":"2024-06-18T13:03:45.475637+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:09:15.038131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.421918+0000","last_change":"2024-06-18T13:02:36.443442+0000","last_active":"2024-06-18T13:03:07.421918+0000","last_peered":"2024-06-18T13:03:07.421918+0000","last_clean":"2024-06-18T13:03:07.421918+0000","last_became_active":"2024-06-18T13:02:36.443133+0000","last_became_peered":"2024-06-18T13:02:36.443133+0000","last_unstale":"2024-06-18T13:03:07.421918+0000","last_undegraded":"2024-06-18T13:03:07.421918+0000","last_fullsized":"2024-06-18T13:03:07.421918+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:36:51.652527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"62'1","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420006+0000","last_change":"2024-06-18T13:02:30.419427+0000","last_active":"2024-06-18T13:03:07.420006+0000","last_peered":"2024-06-18T13:03:07.420006+0000","last_clean":"2024-06-18T13:03:07.420006+0000","last_became_active":"2024-06-18T13:02:30.418293+0000","last_became_peered":"2024-06-18T13:02:30.418293+0000","last_unstale":"2024-06-18T13:03:07.420006+0000","last_undegraded":"2024-06-18T13:03:07.420006+0000","last_fullsized":"2024-06-18T13:03:07.420006+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T21:45:43.288433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"69'11","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444062+0000","last_change":"2024-06-18T13:02:32.422917+0000","last_active":"2024-06-18T13:03:07.444062+0000","last_peered":"2024-06-18T13:03:07.444062+0000","last_clean":"2024-06-18T13:03:07.444062+0000","last_became_active":"2024-06-18T13:02:32.422619+0000","last_became_peered":"2024-06-18T13:02:32.422619+0000","last_unstale":"2024-06-18T13:03:07.444062+0000","last_undegraded":"2024-06-18T13:03:07.444062+0000","last_fullsized":"2024-06-18T13:03:07.444062+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:51:17.241337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444434+0000","last_change":"2024-06-18T13:02:34.438808+0000","last_active":"2024-06-18T13:03:07.444434+0000","last_peered":"2024-06-18T13:03:07.444434+0000","last_clean":"2024-06-18T13:03:07.444434+0000","last_became_active":"2024-06-18T13:02:34.438478+0000","last_became_peered":"2024-06-18T13:02:34.438478+0000","last_unstale":"2024-06-18T13:03:07.444434+0000","last_undegraded":"2024-06-18T13:03:07.444434+0000","last_fullsized":"2024-06-18T13:03:07.444434+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T22:30:48.205792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444971+0000","last_change":"2024-06-18T13:02:36.470203+0000","last_active":"2024-06-18T13:03:07.444971+0000","last_peered":"2024-06-18T13:03:07.444971+0000","last_clean":"2024-06-18T13:03:07.444971+0000","last_became_active":"2024-06-18T13:02:36.469810+0000","last_became_peered":"2024-06-18T13:02:36.469810+0000","last_unstale":"2024-06-18T13:03:07.444971+0000","last_undegraded":"2024-06-18T13:03:07.444971+0000","last_fullsized":"2024-06-18T13:03:07.444971+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:26:15.264159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443680+0000","last_change":"2024-06-18T13:02:30.411860+0000","last_active":"2024-06-18T13:03:07.443680+0000","last_peered":"2024-06-18T13:03:07.443680+0000","last_clean":"2024-06-18T13:03:07.443680+0000","last_became_active":"2024-06-18T13:02:30.411592+0000","last_became_peered":"2024-06-18T13:02:30.411592+0000","last_unstale":"2024-06-18T13:03:07.443680+0000","last_undegraded":"2024-06-18T13:03:07.443680+0000","last_fullsized":"2024-06-18T13:03:07.443680+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T16:14:57.880958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"69'15","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.443669+0000","last_change":"2024-06-18T13:02:32.416596+0000","last_active":"2024-06-18T13:03:07.443669+0000","last_peered":"2024-06-18T13:03:07.443669+0000","last_clean":"2024-06-18T13:03:07.443669+0000","last_became_active":"2024-06-18T13:02:32.416275+0000","last_became_peered":"2024-06-18T13:02:32.416275+0000","last_unstale":"2024-06-18T13:03:07.443669+0000","last_undegraded":"2024-06-18T13:03:07.443669+0000","last_fullsized":"2024-06-18T13:03:07.443669+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T18:35:49.192197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.422189+0000","last_change":"2024-06-18T13:02:34.429610+0000","last_active":"2024-06-18T13:03:07.422189+0000","last_peered":"2024-06-18T13:03:07.422189+0000","last_clean":"2024-06-18T13:03:07.422189+0000","last_became_active":"2024-06-18T13:02:34.429268+0000","last_became_peered":"2024-06-18T13:02:34.429268+0000","last_unstale":"2024-06-18T13:03:07.422189+0000","last_undegraded":"2024-06-18T13:03:07.422189+0000","last_fullsized":"2024-06-18T13:03:07.422189+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:13:31.374986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420270+0000","last_change":"2024-06-18T13:02:36.471398+0000","last_active":"2024-06-18T13:03:07.420270+0000","last_peered":"2024-06-18T13:03:07.420270+0000","last_clean":"2024-06-18T13:03:07.420270+0000","last_became_active":"2024-06-18T13:02:36.471038+0000","last_became_peered":"2024-06-18T13:02:36.471038+0000","last_unstale":"2024-06-18T13:03:07.420270+0000","last_undegraded":"2024-06-18T13:03:07.420270+0000","last_fullsized":"2024-06-18T13:03:07.420270+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:35.421809+0000","last_clean_scrub_stamp":"2024-06-18T13:02:35.421809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T15:30:41.530082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.419924+0000","last_change":"2024-06-18T13:02:30.419110+0000","last_active":"2024-06-18T13:03:07.419924+0000","last_peered":"2024-06-18T13:03:07.419924+0000","last_clean":"2024-06-18T13:03:07.419924+0000","last_became_active":"2024-06-18T13:02:30.418827+0000","last_became_peered":"2024-06-18T13:02:30.418827+0000","last_unstale":"2024-06-18T13:03:07.419924+0000","last_undegraded":"2024-06-18T13:03:07.419924+0000","last_fullsized":"2024-06-18T13:03:07.419924+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:29.383846+0000","last_clean_scrub_stamp":"2024-06-18T13:02:29.383846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:26:13.772453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"69'12","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.420077+0000","last_change":"2024-06-18T13:02:32.423756+0000","last_active":"2024-06-18T13:03:07.420077+0000","last_peered":"2024-06-18T13:03:07.420077+0000","last_clean":"2024-06-18T13:03:07.420077+0000","last_became_active":"2024-06-18T13:02:32.423449+0000","last_became_peered":"2024-06-18T13:02:32.423449+0000","last_unstale":"2024-06-18T13:03:07.420077+0000","last_undegraded":"2024-06-18T13:03:07.420077+0000","last_fullsized":"2024-06-18T13:03:07.420077+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:31.399429+0000","last_clean_scrub_stamp":"2024-06-18T13:02:31.399429+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T13:50:28.551618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-18T13:03:07.444731+0000","last_change":"2024-06-18T13:02:34.439676+0000","last_active":"2024-06-18T13:03:07.444731+0000","last_peered":"2024-06-18T13:03:07.444731+0000","last_clean":"2024-06-18T13:03:07.444731+0000","last_became_active":"2024-06-18T13:02:34.439253+0000","last_became_peered":"2024-06-18T13:02:34.439253+0000","last_unstale":"2024-06-18T13:03:07.444731+0000","last_undegraded":"2024-06-18T13:03:07.444731+0000","last_fullsized":"2024-06-18T13:03:07.444731+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-18T13:02:33.414546+0000","last_clean_scrub_stamp":"2024-06-18T13:02:33.414546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-19T17:03:14.794520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":56,"seq":240518168599,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32532,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712620,"statfs":{"total":95995035648,"available":95961722880,"internally_reserved":0,"allocated":1388544,"data_stored":916249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462939,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32528,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712624,"statfs":{"total":95995035648,"available":95961726976,"internally_reserved":0,"allocated":1384448,"data_stored":917014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724576,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31940,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713212,"statfs":{"total":95995035648,"available":95962329088,"internally_reserved":0,"allocated":778240,"data_stored":326088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":794624,"data_stored":325295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149737,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":815104,"data_stored":325659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27784,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717368,"statfs":{"total":95995035648,"available":95966584832,"internally_reserved":0,"allocated":782336,"data_stored":326715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":827392,"data_stored":326456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712620,"statfs":{"total":95995035648,"available":95961722880,"internally_reserved":0,"allocated":1384448,"data_stored":915509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-18T13:03:56.491 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-18T13:03:56.491 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-18T13:03:56.491 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-18T13:03:56.491 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00 shell --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 -- ceph health --format=json 2024-06-18T13:03:56.742 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/mon.a/config 2024-06-18T13:03:57.244 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:57 smithi179 ceph-mon[33312]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:57.244 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:57 smithi179 ceph-mon[33312]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-18T13:03:57.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:57 smithi167 ceph-mon[29496]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:57.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:57 smithi167 ceph-mon[29496]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-18T13:03:57.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:57 smithi167 ceph-mon[37701]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:03:57.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:57 smithi167 ceph-mon[37701]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-18T13:03:58.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.279+0000 7fb491cd4640 1 -- 172.21.15.167:0/3753114324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb48c1055c0 msgr2=0x7fb48c10d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:58.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.279+0000 7fb491cd4640 1 --2- 172.21.15.167:0/3753114324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb48c1055c0 0x7fb48c10d140 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb484009ca0 tx=0x7fb48402a8c0 comp rx=0 tx=0).stop 2024-06-18T13:03:58.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.279+0000 7fb491cd4640 1 -- 172.21.15.167:0/3753114324 shutdown_connections 2024-06-18T13:03:58.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.279+0000 7fb491cd4640 1 --2- 172.21.15.167:0/3753114324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb48c1055c0 0x7fb48c10d140 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:58.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.279+0000 7fb491cd4640 1 --2- 172.21.15.167:0/3753114324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb48c100e40 0x7fb48c104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:58.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.279+0000 7fb491cd4640 1 --2- 172.21.15.167:0/3753114324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb48c100490 0x7fb48c100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:58.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.279+0000 7fb491cd4640 1 -- 172.21.15.167:0/3753114324 >> 172.21.15.167:0/3753114324 conn(0x7fb48c0fbf50 msgr2=0x7fb48c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:58.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.279+0000 7fb491cd4640 1 -- 172.21.15.167:0/3753114324 shutdown_connections 2024-06-18T13:03:58.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.279+0000 7fb491cd4640 1 -- 172.21.15.167:0/3753114324 wait complete. 2024-06-18T13:03:58.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.280+0000 7fb491cd4640 1 Processor -- start 2024-06-18T13:03:58.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.280+0000 7fb491cd4640 1 -- start start 2024-06-18T13:03:58.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.280+0000 7fb491cd4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb48c100490 0x7fb48c1970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:58.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.280+0000 7fb491cd4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb48c100e40 0x7fb48c197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:58.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.280+0000 7fb491cd4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb48c1055c0 0x7fb48c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:58.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.280+0000 7fb491cd4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c110bf0 con 0x7fb48c1055c0 2024-06-18T13:03:58.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.280+0000 7fb491cd4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c110a70 con 0x7fb48c100e40 2024-06-18T13:03:58.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.280+0000 7fb491cd4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb48c110d70 con 0x7fb48c100490 2024-06-18T13:03:58.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.280+0000 7fb48b7fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb48c100490 0x7fb48c1970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:58.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.280+0000 7fb48bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb48c1055c0 0x7fb48c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:58.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.280+0000 7fb48b7fe640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb48c100490 0x7fb48c1970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:50142/0 (socket says 172.21.15.167:50142) 2024-06-18T13:03:58.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.280+0000 7fb48b7fe640 1 -- 172.21.15.167:0/4291516380 learned_addr learned my addr 172.21.15.167:0/4291516380 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:03:58.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.280+0000 7fb483fff640 1 --2- 172.21.15.167:0/4291516380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb48c100e40 0x7fb48c197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:58.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.281+0000 7fb48b7fe640 1 -- 172.21.15.167:0/4291516380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb48c100e40 msgr2=0x7fb48c197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:58.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.281+0000 7fb48b7fe640 1 --2- 172.21.15.167:0/4291516380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb48c100e40 0x7fb48c197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:58.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.281+0000 7fb48b7fe640 1 -- 172.21.15.167:0/4291516380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb48c1055c0 msgr2=0x7fb48c19b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:58.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.281+0000 7fb48b7fe640 1 --2- 172.21.15.167:0/4291516380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb48c1055c0 0x7fb48c19b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:58.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.281+0000 7fb48b7fe640 1 -- 172.21.15.167:0/4291516380 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48c19c080 con 0x7fb48c100490 2024-06-18T13:03:58.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.281+0000 7fb48bfff640 1 --2- 172.21.15.167:0/4291516380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb48c1055c0 0x7fb48c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:03:58.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.281+0000 7fb48b7fe640 1 --2- 172.21.15.167:0/4291516380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb48c100490 0x7fb48c1970d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00e9f0 tx=0x7fb47c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:58.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.281+0000 7fb4897fa640 1 -- 172.21.15.167:0/4291516380 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c00cd80 con 0x7fb48c100490 2024-06-18T13:03:58.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.281+0000 7fb4897fa640 1 -- 172.21.15.167:0/4291516380 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fb47c0108b0 con 0x7fb48c100490 2024-06-18T13:03:58.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.281+0000 7fb491cd4640 1 -- 172.21.15.167:0/4291516380 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48c19c370 con 0x7fb48c100490 2024-06-18T13:03:58.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.281+0000 7fb4897fa640 1 -- 172.21.15.167:0/4291516380 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c018830 con 0x7fb48c100490 2024-06-18T13:03:58.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.281+0000 7fb491cd4640 1 -- 172.21.15.167:0/4291516380 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48c1a3c50 con 0x7fb48c100490 2024-06-18T13:03:58.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.282+0000 7fb491cd4640 1 -- 172.21.15.167:0/4291516380 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb48c100870 con 0x7fb48c100490 2024-06-18T13:03:58.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.283+0000 7fb4897fa640 1 -- 172.21.15.167:0/4291516380 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7fb47c010430 con 0x7fb48c100490 2024-06-18T13:03:58.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.283+0000 7fb4897fa640 1 --2- 172.21.15.167:0/4291516380 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb47806e040 0x7fb478070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:03:58.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.283+0000 7fb483fff640 1 --2- 172.21.15.167:0/4291516380 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb47806e040 0x7fb478070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:03:58.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.283+0000 7fb4897fa640 1 -- 172.21.15.167:0/4291516380 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 7421+0+0 (secure 0 0 0) 0x7fb47c08ee00 con 0x7fb48c100490 2024-06-18T13:03:58.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.284+0000 7fb483fff640 1 --2- 172.21.15.167:0/4291516380 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb47806e040 0x7fb478070500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb4740095f0 tx=0x7fb474005f90 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:03:58.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.287+0000 7fb4897fa640 1 -- 172.21.15.167:0/4291516380 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fb47c05fff0 con 0x7fb48c100490 2024-06-18T13:03:58.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.505+0000 7fb491cd4640 1 -- 172.21.15.167:0/4291516380 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb48c108800 con 0x7fb48c100490 2024-06-18T13:03:58.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.506+0000 7fb4897fa640 1 -- 172.21.15.167:0/4291516380 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb47c064820 con 0x7fb48c100490 2024-06-18T13:03:58.506 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-18T13:03:58.506 INFO:teuthology.orchestra.run.smithi167.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-18T13:03:58.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.507+0000 7fb491cd4640 1 -- 172.21.15.167:0/4291516380 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb47806e040 msgr2=0x7fb478070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:58.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.507+0000 7fb491cd4640 1 --2- 172.21.15.167:0/4291516380 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb47806e040 0x7fb478070500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb4740095f0 tx=0x7fb474005f90 comp rx=0 tx=0).stop 2024-06-18T13:03:58.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.508+0000 7fb491cd4640 1 -- 172.21.15.167:0/4291516380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb48c100490 msgr2=0x7fb48c1970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:03:58.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.508+0000 7fb491cd4640 1 --2- 172.21.15.167:0/4291516380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb48c100490 0x7fb48c1970d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00e9f0 tx=0x7fb47c00eeb0 comp rx=0 tx=0).stop 2024-06-18T13:03:58.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.508+0000 7fb491cd4640 1 -- 172.21.15.167:0/4291516380 shutdown_connections 2024-06-18T13:03:58.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.508+0000 7fb491cd4640 1 --2- 172.21.15.167:0/4291516380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb48c1055c0 0x7fb48c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:58.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.508+0000 7fb491cd4640 1 --2- 172.21.15.167:0/4291516380 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb47806e040 0x7fb478070500 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:58.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.508+0000 7fb491cd4640 1 --2- 172.21.15.167:0/4291516380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb48c100e40 0x7fb48c197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:58.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.508+0000 7fb491cd4640 1 --2- 172.21.15.167:0/4291516380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb48c100490 0x7fb48c1970d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:03:58.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.508+0000 7fb491cd4640 1 -- 172.21.15.167:0/4291516380 >> 172.21.15.167:0/4291516380 conn(0x7fb48c0fbf50 msgr2=0x7fb48c0fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:03:58.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.508+0000 7fb491cd4640 1 -- 172.21.15.167:0/4291516380 shutdown_connections 2024-06-18T13:03:58.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-18T13:03:58.508+0000 7fb491cd4640 1 -- 172.21.15.167:0/4291516380 wait complete. 2024-06-18T13:03:58.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:03:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:03:59.186 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:03:58 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[71439]: ts=2024-06-18T13:03:58.923Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002958358s 2024-06-18T13:03:59.212 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 systemd[1]: Started Ceph grafana.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:03:59.214 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-18T13:03:59.214 INFO:tasks.cephadm:Setup complete, yielding 2024-06-18T13:03:59.214 INFO:teuthology.run_tasks:Running task workunit... 2024-06-18T13:03:59.230 INFO:tasks.workunit:Pulling workunits from ref f18f3f1cbd11e4a41c557e616d172f24deb91473 2024-06-18T13:03:59.231 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-18T13:03:59.231 DEBUG:teuthology.orchestra.run.smithi167:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-18T13:03:59.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-18T13:03:59.247 INFO:teuthology.orchestra.run.smithi167.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-06-18T13:03:59.248 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-06-18T13:03:59.303 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-06-18T13:03:59.303 DEBUG:teuthology.orchestra.run.smithi167:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-06-18T13:03:59.358 INFO:tasks.workunit:timeout=3h 2024-06-18T13:03:59.358 INFO:tasks.workunit:cleanup=True 2024-06-18T13:03:59.359 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f18f3f1cbd11e4a41c557e616d172f24deb91473 2024-06-18T13:03:59.414 INFO:tasks.workunit.client.0.smithi167.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-18T13:03:59.464 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-mon[33312]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:59.464 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4291516380' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-18T13:03:59.464 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226716798Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-06-18T13:03:59.464 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226935865Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-06-18T13:03:59.464 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226949075Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-06-18T13:03:59.464 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226952576Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-06-18T13:03:59.464 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226955393Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-06-18T13:03:59.465 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226958128Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-06-18T13:03:59.465 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226960745Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-06-18T13:03:59.465 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226963206Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-06-18T13:03:59.465 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226965854Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-06-18T13:03:59.465 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226968635Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-06-18T13:03:59.465 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.2269716Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-06-18T13:03:59.465 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226974449Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-06-18T13:03:59.465 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226977037Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-06-18T13:03:59.465 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226979567Z level=info msg="Path Home" path=/usr/share/grafana 2024-06-18T13:03:59.465 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226982215Z level=info msg="Path Data" path=/var/lib/grafana 2024-06-18T13:03:59.465 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226985036Z level=info msg="Path Logs" path=/var/log/grafana 2024-06-18T13:03:59.466 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226987523Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-06-18T13:03:59.466 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226989993Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-06-18T13:03:59.466 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=settings t=2024-06-18T13:03:59.226992549Z level=info msg="App mode production" 2024-06-18T13:03:59.467 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=sqlstore t=2024-06-18T13:03:59.227032833Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-06-18T13:03:59.467 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=sqlstore t=2024-06-18T13:03:59.227043447Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-06-18T13:03:59.467 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.227896449Z level=info msg="Starting DB migrations" 2024-06-18T13:03:59.467 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.227961986Z level=info msg="Executing migration" id="create migration_log table" 2024-06-18T13:03:59.467 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.228531665Z level=info msg="Executing migration" id="create user table" 2024-06-18T13:03:59.467 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.228975078Z level=info msg="Executing migration" id="add unique index user.login" 2024-06-18T13:03:59.467 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.22945238Z level=info msg="Executing migration" id="add unique index user.email" 2024-06-18T13:03:59.467 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.229891768Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-06-18T13:03:59.467 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.230464746Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-06-18T13:03:59.467 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.230847929Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-06-18T13:03:59.468 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.231706772Z level=info msg="Executing migration" id="create user table v2" 2024-06-18T13:03:59.468 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.232109453Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-06-18T13:03:59.468 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.232488445Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-06-18T13:03:59.468 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.232920865Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-06-18T13:03:59.468 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.233275185Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-06-18T13:03:59.468 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.233673508Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-06-18T13:03:59.468 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.23413925Z level=info msg="Executing migration" id="Update user table charset" 2024-06-18T13:03:59.468 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.234449672Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-06-18T13:03:59.468 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.234843517Z level=info msg="Executing migration" id="Add missing user data" 2024-06-18T13:03:59.468 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.235218708Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-06-18T13:03:59.468 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.235618738Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-06-18T13:03:59.468 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.236038906Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-06-18T13:03:59.468 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.236464375Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-06-18T13:03:59.469 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.237872176Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-06-18T13:03:59.469 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.238353146Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-06-18T13:03:59.469 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.238815555Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-06-18T13:03:59.469 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.23924548Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-06-18T13:03:59.469 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.239653111Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-06-18T13:03:59.469 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.240049579Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-06-18T13:03:59.469 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.240336856Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-06-18T13:03:59.469 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.240716326Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-06-18T13:03:59.469 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.241102511Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-06-18T13:03:59.469 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.241477214Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-06-18T13:03:59.469 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.241832976Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-06-18T13:03:59.469 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.242625273Z level=info msg="Executing migration" id="create temp_user v2" 2024-06-18T13:03:59.469 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.243029032Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-06-18T13:03:59.469 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.243404042Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-06-18T13:03:59.470 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.243748774Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-06-18T13:03:59.470 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.244131203Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-06-18T13:03:59.470 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.244517164Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-06-18T13:03:59.470 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.244808268Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-06-18T13:03:59.470 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.245150032Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-06-18T13:03:59.470 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.245444346Z level=info msg="Executing migration" id="create star table" 2024-06-18T13:03:59.470 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.245768205Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-06-18T13:03:59.470 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.246184856Z level=info msg="Executing migration" id="create org table v1" 2024-06-18T13:03:59.470 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.246559169Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-06-18T13:03:59.470 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.248884793Z level=info msg="Executing migration" id="create org_user table v1" 2024-06-18T13:03:59.470 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.249278014Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-06-18T13:03:59.471 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.249800792Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-06-18T13:03:59.471 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.250425404Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-06-18T13:03:59.471 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.250840613Z level=info msg="Executing migration" id="Update org table charset" 2024-06-18T13:03:59.471 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.253574336Z level=info msg="Executing migration" id="Update org_user table charset" 2024-06-18T13:03:59.471 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.256364774Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-06-18T13:03:59.471 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.256689467Z level=info msg="Executing migration" id="create dashboard table" 2024-06-18T13:03:59.471 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.259458716Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-06-18T13:03:59.471 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.264821283Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-06-18T13:03:59.471 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.265279435Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-06-18T13:03:59.471 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.268008766Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-06-18T13:03:59.471 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.270648043Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-06-18T13:03:59.471 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.271082331Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-06-18T13:03:59.471 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.273901324Z level=info msg="Executing migration" id="create dashboard v2" 2024-06-18T13:03:59.472 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.276602009Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-06-18T13:03:59.472 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.277280786Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-06-18T13:03:59.472 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.277741226Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-06-18T13:03:59.472 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.278540543Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-06-18T13:03:59.472 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.279428825Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-06-18T13:03:59.472 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.280143709Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-06-18T13:03:59.472 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.280740807Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-06-18T13:03:59.472 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.281525499Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-06-18T13:03:59.472 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.282319414Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-06-18T13:03:59.472 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.283060431Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-06-18T13:03:59.472 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.283551113Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-06-18T13:03:59.472 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.284339524Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-06-18T13:03:59.473 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.284758079Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-06-18T13:03:59.473 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.285219398Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-06-18T13:03:59.473 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.286058544Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-06-18T13:03:59.473 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.286674826Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-06-18T13:03:59.473 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.287179486Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-06-18T13:03:59.473 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.287994951Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-06-18T13:03:59.473 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.288803157Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-06-18T13:03:59.473 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.289113792Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-06-18T13:03:59.473 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.290028924Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-06-18T13:03:59.473 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.290738248Z level=info msg="Executing migration" id="Update dashboard title length" 2024-06-18T13:03:59.473 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.291729753Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-06-18T13:03:59.474 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.292424082Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-06-18T13:03:59.474 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.293338075Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-06-18T13:03:59.474 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.294429342Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-06-18T13:03:59.474 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.295334831Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-06-18T13:03:59.474 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.295801072Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-06-18T13:03:59.474 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.296676102Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-06-18T13:03:59.474 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.297503753Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-06-18T13:03:59.474 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.298363083Z level=info msg="Executing migration" id="Add check_sum column" 2024-06-18T13:03:59.474 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.298892803Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-06-18T13:03:59.474 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.299883261Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-06-18T13:03:59.474 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.300608226Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-06-18T13:03:59.474 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.301436043Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-06-18T13:03:59.475 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.301886117Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-06-18T13:03:59.475 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.302455988Z level=info msg="Executing migration" id="create data_source table" 2024-06-18T13:03:59.475 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.305124101Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-06-18T13:03:59.475 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.307900218Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.308331212Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.308724131Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.309148098Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.311657403Z level=info msg="Executing migration" id="create data_source table v2" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.314385285Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.316955949Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.317379706Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.320129758Z level=info msg="Executing migration" id="Add column with_credentials" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.320753288Z level=info msg="Executing migration" id="Add secure json data column" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.322785675Z level=info msg="Executing migration" id="Update data_source table charset" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.325692351Z level=info msg="Executing migration" id="Update initial version to 1" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.326041281Z level=info msg="Executing migration" id="Add read_only data column" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.328917145Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.331432179Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-06-18T13:03:59.476 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.334009504Z level=info msg="Executing migration" id="Add uid column" 2024-06-18T13:03:59.477 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.334638819Z level=info msg="Executing migration" id="Update uid value" 2024-06-18T13:03:59.477 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.337210231Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-06-18T13:03:59.477 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.337681363Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-06-18T13:03:59.477 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.340152119Z level=info msg="Executing migration" id="create api_key table" 2024-06-18T13:03:59.477 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.342784093Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-06-18T13:03:59.477 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.343232985Z level=info msg="Executing migration" id="add index api_key.key" 2024-06-18T13:03:59.477 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.345909081Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-06-18T13:03:59.477 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.348631203Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-06-18T13:03:59.477 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.35146357Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-06-18T13:03:59.477 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.354221982Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-06-18T13:03:59.477 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.35675962Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-06-18T13:03:59.477 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.359648338Z level=info msg="Executing migration" id="create api_key table v2" 2024-06-18T13:03:59.477 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.36230147Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-06-18T13:03:59.477 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.364990407Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-06-18T13:03:59.478 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.367596768Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-06-18T13:03:59.478 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.368483382Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-06-18T13:03:59.478 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.369402674Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-06-18T13:03:59.478 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.370188249Z level=info msg="Executing migration" id="Update api_key table charset" 2024-06-18T13:03:59.478 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.37109935Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-06-18T13:03:59.478 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.371845471Z level=info msg="Executing migration" id="Add service account foreign key" 2024-06-18T13:03:59.478 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.372772353Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-06-18T13:03:59.478 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.373498551Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-06-18T13:03:59.478 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.374338181Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-06-18T13:03:59.478 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.375217119Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-06-18T13:03:59.478 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.376030369Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-06-18T13:03:59.478 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.376910922Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-06-18T13:03:59.478 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.377722373Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-06-18T13:03:59.478 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.37857679Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-06-18T13:03:59.479 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.37944023Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-06-18T13:03:59.479 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.380350817Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-06-18T13:03:59.479 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.381171981Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-06-18T13:03:59.479 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.382101984Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-06-18T13:03:59.479 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.382871109Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-06-18T13:03:59.479 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.383775356Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-06-18T13:03:59.479 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.384737325Z level=info msg="Executing migration" id="create quota table v1" 2024-06-18T13:03:59.479 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.385510749Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-06-18T13:03:59.479 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.38636341Z level=info msg="Executing migration" id="Update quota table charset" 2024-06-18T13:03:59.479 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.387308592Z level=info msg="Executing migration" id="create plugin_setting table" 2024-06-18T13:03:59.479 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.388099904Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-06-18T13:03:59.479 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.389079019Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-06-18T13:03:59.479 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.389864443Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-06-18T13:03:59.480 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.390727209Z level=info msg="Executing migration" id="create session table" 2024-06-18T13:03:59.480 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.391740002Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-06-18T13:03:59.480 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.392601188Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-06-18T13:03:59.480 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.393337796Z level=info msg="Executing migration" id="create playlist table v2" 2024-06-18T13:03:59.480 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.394362499Z level=info msg="Executing migration" id="create playlist item table v2" 2024-06-18T13:03:59.480 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.395071636Z level=info msg="Executing migration" id="Update playlist table charset" 2024-06-18T13:03:59.480 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.396043255Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-06-18T13:03:59.480 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.396763573Z level=info msg="Executing migration" id="drop preferences table v2" 2024-06-18T13:03:59.480 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.397760187Z level=info msg="Executing migration" id="drop preferences table v3" 2024-06-18T13:03:59.480 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.400638946Z level=info msg="Executing migration" id="create preferences table v3" 2024-06-18T13:03:59.480 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.40336606Z level=info msg="Executing migration" id="Update preferences table charset" 2024-06-18T13:03:59.480 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.406169793Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-06-18T13:03:59.480 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.409047215Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-06-18T13:03:59.480 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.411992419Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-06-18T13:03:59.481 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.414809802Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-06-18T13:03:59.481 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.41767648Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-06-18T13:03:59.481 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.420462201Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-06-18T13:03:59.481 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.423236344Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-06-18T13:03:59.481 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.425980071Z level=info msg="Executing migration" id="create alert table v1" 2024-06-18T13:03:59.481 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.429090456Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-06-18T13:03:59.481 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.431717772Z level=info msg="Executing migration" id="add index alert state" 2024-06-18T13:03:59.481 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.434394541Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-06-18T13:03:59.481 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.437261179Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-06-18T13:03:59.481 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.440126766Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-06-18T13:03:59.481 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.44305677Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-06-18T13:03:59.481 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.445704971Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-06-18T13:03:59.481 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.448491992Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-06-18T13:03:59.481 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.451198243Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-06-18T13:03:59.482 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.454075841Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-06-18T13:03:59.482 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.456836134Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-06-18T13:03:59.482 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.459656664Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-06-18T13:03:59.482 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.462290275Z level=info msg="Executing migration" id="Add column is_default" 2024-06-18T13:03:59.482 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.46503885Z level=info msg="Executing migration" id="Add column frequency" 2024-06-18T13:03:59.482 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.467593265Z level=info msg="Executing migration" id="Add column send_reminder" 2024-06-18T13:03:59.482 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.470209054Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-06-18T13:03:59.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:59 smithi167 ceph-mon[37701]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:59.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:03:59 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4291516380' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-18T13:03:59.483 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:59 smithi167 ceph-mon[29496]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:03:59.483 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:03:59 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4291516380' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-18T13:03:59.716 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.472865637Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-06-18T13:03:59.716 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.475610134Z level=info msg="Executing migration" id="Update alert table charset" 2024-06-18T13:03:59.717 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.478279037Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-06-18T13:03:59.717 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.480974918Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-06-18T13:03:59.717 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.48380978Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-06-18T13:03:59.717 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.486455402Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-06-18T13:03:59.717 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.489058891Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-06-18T13:03:59.717 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.491744255Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-06-18T13:03:59.717 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.49438835Z level=info msg="Executing migration" id="Add for to alert table" 2024-06-18T13:03:59.717 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.495230306Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-06-18T13:03:59.717 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.4960872Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-06-18T13:03:59.717 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.496996061Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-06-18T13:03:59.717 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.497832868Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-06-18T13:03:59.717 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.498749994Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-06-18T13:03:59.718 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.49952525Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-06-18T13:03:59.718 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.500494681Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-06-18T13:03:59.718 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.501179774Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-06-18T13:03:59.718 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.502078246Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-06-18T13:03:59.718 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.502873769Z level=info msg="Executing migration" id="create annotation table v5" 2024-06-18T13:03:59.718 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.503885676Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-06-18T13:03:59.718 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.504528127Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-06-18T13:03:59.718 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.505307992Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-06-18T13:03:59.718 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.506196873Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-06-18T13:03:59.718 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.507051159Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-06-18T13:03:59.718 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.507981166Z level=info msg="Executing migration" id="Update annotation table charset" 2024-06-18T13:03:59.719 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.508720678Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-06-18T13:03:59.719 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.50958325Z level=info msg="Executing migration" id="Drop category_id index" 2024-06-18T13:03:59.719 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.510458492Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-06-18T13:03:59.719 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.511479439Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-06-18T13:03:59.719 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.512225668Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-06-18T13:03:59.719 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.513055306Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-06-18T13:03:59.719 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.513973041Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-06-18T13:03:59.719 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.515955958Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-06-18T13:03:59.719 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.5168141Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-06-18T13:03:59.719 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.517668519Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-06-18T13:03:59.719 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.5185197Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-06-18T13:03:59.719 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.519426103Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-06-18T13:03:59.721 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.520404211Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-06-18T13:03:59.721 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.521381635Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-06-18T13:03:59.721 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.522176219Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-06-18T13:03:59.721 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.523140493Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-06-18T13:03:59.721 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.523922738Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-06-18T13:03:59.721 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.524903144Z level=info msg="Executing migration" id="Add epoch_end column" 2024-06-18T13:03:59.721 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.52573435Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-06-18T13:03:59.721 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.528661096Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-06-18T13:03:59.721 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.531398629Z level=info msg="Executing migration" id="Move region to single row" 2024-06-18T13:03:59.721 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.534401118Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-06-18T13:03:59.722 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.537110363Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-06-18T13:03:59.722 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.53999245Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-06-18T13:03:59.722 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.540478175Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-06-18T13:03:59.722 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.543171298Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-06-18T13:03:59.722 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.543687876Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-06-18T13:03:59.722 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.545987158Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-06-18T13:03:59.722 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.548670211Z level=info msg="Executing migration" id="create test_data table" 2024-06-18T13:03:59.722 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.551396141Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-06-18T13:03:59.722 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.55183775Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-06-18T13:03:59.722 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.554648175Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-06-18T13:03:59.722 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.557254277Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-06-18T13:03:59.722 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.559878856Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-06-18T13:03:59.723 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.562730525Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-06-18T13:03:59.723 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.565353402Z level=info msg="Executing migration" id="create team table" 2024-06-18T13:03:59.723 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.568093022Z level=info msg="Executing migration" id="add index team.org_id" 2024-06-18T13:03:59.723 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.570833737Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-06-18T13:03:59.723 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.571663531Z level=info msg="Executing migration" id="create team member table" 2024-06-18T13:03:59.723 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.572325899Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-06-18T13:03:59.723 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.573268911Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-06-18T13:03:59.723 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.573957024Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-06-18T13:03:59.723 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.57477708Z level=info msg="Executing migration" id="Add column email to team table" 2024-06-18T13:03:59.723 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.575639138Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-06-18T13:03:59.723 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.576480011Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-06-18T13:03:59.723 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.57730529Z level=info msg="Executing migration" id="create dashboard acl table" 2024-06-18T13:03:59.723 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.578099689Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-06-18T13:03:59.723 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.578916172Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-06-18T13:03:59.724 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.579380101Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-06-18T13:03:59.724 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.579833508Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-06-18T13:03:59.724 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.580779593Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-06-18T13:03:59.724 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.581426944Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-06-18T13:03:59.724 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.58191224Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-06-18T13:03:59.724 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.582865764Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-06-18T13:03:59.724 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.583512842Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-06-18T13:03:59.724 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.584278257Z level=info msg="Executing migration" id="create tag table" 2024-06-18T13:03:59.724 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.584735206Z level=info msg="Executing migration" id="add index tag.key_value" 2024-06-18T13:03:59.724 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.585172447Z level=info msg="Executing migration" id="create login attempt table" 2024-06-18T13:03:59.724 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.586012938Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-06-18T13:03:59.724 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.586999097Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-06-18T13:03:59.724 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.587730234Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-06-18T13:03:59.724 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.590127577Z level=info msg="Executing migration" id="create login_attempt v2" 2024-06-18T13:03:59.725 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.591057408Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-06-18T13:03:59.725 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.591960493Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-06-18T13:03:59.725 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.592663897Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-06-18T13:03:59.725 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.593066416Z level=info msg="Executing migration" id="create user auth table" 2024-06-18T13:03:59.725 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.593913587Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-06-18T13:03:59.725 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.594777214Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-06-18T13:03:59.725 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.595709136Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-06-18T13:03:59.725 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.59657801Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-06-18T13:03:59.725 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.597506009Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-06-18T13:03:59.725 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.598480611Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-06-18T13:03:59.725 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.599424415Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-06-18T13:03:59.725 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.599904342Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-06-18T13:03:59.726 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.602675953Z level=info msg="Executing migration" id="create server_lock table" 2024-06-18T13:03:59.726 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.60536225Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-06-18T13:03:59.726 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.60820815Z level=info msg="Executing migration" id="create user auth token table" 2024-06-18T13:03:59.726 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.608691509Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-06-18T13:03:59.726 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.611533456Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-06-18T13:03:59.726 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.614320426Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-06-18T13:03:59.726 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.617105101Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-06-18T13:03:59.726 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.618155779Z level=info msg="Executing migration" id="create cache_data table" 2024-06-18T13:03:59.726 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.620823807Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-06-18T13:03:59.726 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.62345518Z level=info msg="Executing migration" id="create short_url table v1" 2024-06-18T13:03:59.726 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.626178239Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-06-18T13:03:59.726 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.626607119Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-06-18T13:03:59.726 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.626873354Z level=info msg="Executing migration" id="delete alert_definition table" 2024-06-18T13:03:59.726 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.629318659Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-06-18T13:03:59.727 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.629768568Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-06-18T13:03:59.727 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.630230442Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-06-18T13:03:59.727 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.630636978Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-06-18T13:03:59.727 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.630896041Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-06-18T13:03:59.727 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.631349786Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-06-18T13:03:59.727 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.631725251Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-06-18T13:03:59.727 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.632281504Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-06-18T13:03:59.727 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.632726485Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-06-18T13:03:59.727 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.633641319Z level=info msg="Executing migration" id="drop alert_definition table" 2024-06-18T13:03:59.727 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.63408497Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-06-18T13:03:59.727 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.634384268Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-06-18T13:03:59.727 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.634761301Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-06-18T13:03:59.727 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.635230044Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-06-18T13:03:59.728 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.635661554Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-06-18T13:03:59.728 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.635928022Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-06-18T13:03:59.728 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.636435709Z level=info msg="Executing migration" id="create alert_instance table" 2024-06-18T13:03:59.728 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.63682644Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-06-18T13:03:59.728 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.6372882Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-06-18T13:03:59.728 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.637728496Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-06-18T13:03:59.728 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.638634143Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-06-18T13:03:59.728 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.639075166Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-06-18T13:03:59.728 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.639488666Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-06-18T13:03:59.728 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.64392793Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-06-18T13:03:59.728 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.64825174Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-06-18T13:03:59.728 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.648710184Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-06-18T13:03:59.728 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.649149534Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-06-18T13:03:59.728 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.650109455Z level=info msg="Executing migration" id="create alert_rule table" 2024-06-18T13:03:59.729 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.650542847Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-06-18T13:03:59.729 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.650977079Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-06-18T13:03:59.729 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.65141566Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-06-18T13:03:59.729 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.651900606Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-06-18T13:03:59.729 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.652200609Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-06-18T13:03:59.729 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.653158013Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-06-18T13:03:59.729 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.654084971Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-06-18T13:03:59.729 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.654983133Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-06-18T13:03:59.729 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.655410832Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-06-18T13:03:59.729 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.655833494Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-06-18T13:03:59.729 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.65679652Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-06-18T13:03:59.729 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.657717526Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-06-18T13:03:59.729 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.658166762Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-06-18T13:03:59.730 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.659082625Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-06-18T13:03:59.730 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.660007046Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-06-18T13:03:59.730 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.660312144Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-06-18T13:03:59.730 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.660764695Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-06-18T13:03:59.730 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.661204889Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-06-18T13:03:59.730 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.661640767Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-06-18T13:03:59.730 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.661913452Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-06-18T13:03:59.730 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.662928611Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-06-18T13:03:59.730 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.664800876Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-06-18T13:03:59.730 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.665877231Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-06-18T13:03:59.730 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.668553733Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-06-18T13:03:59.730 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.671041748Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-06-18T13:03:59.730 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.673726554Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-06-18T13:03:59.731 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.67417457Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-06-18T13:03:59.731 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.676818866Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-06-18T13:03:59.731 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.679448189Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-06-18T13:03:59.731 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.682124985Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-06-18T13:03:59.731 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.682609537Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-06-18T13:03:59.731 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.68534802Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-06-18T13:03:59.731 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.688014856Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-06-18T13:03:59.731 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.690644402Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-06-18T13:03:59.731 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.691659632Z level=info msg="Executing migration" id="create provenance_type table" 2024-06-18T13:03:59.731 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.694463332Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-06-18T13:03:59.731 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.697008992Z level=info msg="Executing migration" id="create alert_image table" 2024-06-18T13:03:59.731 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.69984583Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-06-18T13:03:59.731 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.700345479Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-06-18T13:03:59.731 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.703066853Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-06-18T13:03:59.732 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.703467746Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-06-18T13:03:59.732 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.703870493Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-06-18T13:03:59.732 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.703948101Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-06-18T13:03:59.732 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.704222828Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-06-18T13:03:59.732 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.704636173Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-06-18T13:03:59.732 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.705092991Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-06-18T13:03:59.732 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.705214788Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-06-18T13:03:59.732 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.705321164Z level=info msg="alerts found to migrate" alerts=0 2024-06-18T13:03:59.732 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.705713957Z level=info msg="Executing migration" id="create library_element table v1" 2024-06-18T13:03:59.732 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.706567183Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-06-18T13:03:59.732 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.707307032Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-06-18T13:03:59.732 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.708186488Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-06-18T13:03:59.732 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.708801863Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-06-18T13:03:59.732 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.711612372Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-06-18T13:03:59.733 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.714308887Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-06-18T13:03:59.733 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.717098924Z level=info msg="Executing migration" id="create data_keys table" 2024-06-18T13:03:59.733 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.717581684Z level=info msg="Executing migration" id="create secrets table" 2024-06-18T13:03:59.733 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.720437723Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-06-18T13:04:00.137 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.725978626Z level=info msg="Executing migration" id="add name column into data_keys" 2024-06-18T13:04:00.137 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.728682052Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-06-18T13:04:00.137 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.731674672Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-06-18T13:04:00.137 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.737438052Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-06-18T13:04:00.137 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.74304811Z level=info msg="Executing migration" id="create kv_store table v1" 2024-06-18T13:04:00.137 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.745804276Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-06-18T13:04:00.137 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.74627648Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-06-18T13:04:00.138 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.749010416Z level=info msg="Executing migration" id="create permission table" 2024-06-18T13:04:00.138 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.74944171Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-06-18T13:04:00.138 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.751631796Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-06-18T13:04:00.138 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.75434381Z level=info msg="Executing migration" id="create role table" 2024-06-18T13:04:00.138 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.754862699Z level=info msg="Executing migration" id="add column display_name" 2024-06-18T13:04:00.138 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.757597857Z level=info msg="Executing migration" id="add column group_name" 2024-06-18T13:04:00.138 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.760385498Z level=info msg="Executing migration" id="add index role.org_id" 2024-06-18T13:04:00.138 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.762982568Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-06-18T13:04:00.138 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.763432868Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-06-18T13:04:00.138 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.766346428Z level=info msg="Executing migration" id="create team role table" 2024-06-18T13:04:00.138 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.76902673Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-06-18T13:04:00.138 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.77190234Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-06-18T13:04:00.139 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.774562021Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-06-18T13:04:00.139 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.777411107Z level=info msg="Executing migration" id="create user role table" 2024-06-18T13:04:00.139 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.779999901Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-06-18T13:04:00.139 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.782825954Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-06-18T13:04:00.139 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.78346989Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-06-18T13:04:00.139 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.784446961Z level=info msg="Executing migration" id="create builtin role table" 2024-06-18T13:04:00.139 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.785172021Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-06-18T13:04:00.139 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.785998006Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-06-18T13:04:00.139 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.786457054Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-06-18T13:04:00.140 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.787585819Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-06-18T13:04:00.140 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.788420023Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-06-18T13:04:00.140 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.789274769Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-06-18T13:04:00.140 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.789702266Z level=info msg="Executing migration" id="add unique index role.uid" 2024-06-18T13:04:00.140 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.790481105Z level=info msg="Executing migration" id="create seed assignment table" 2024-06-18T13:04:00.140 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.791303004Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-06-18T13:04:00.140 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.792181385Z level=info msg="Executing migration" id="add column hidden to role table" 2024-06-18T13:04:00.140 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.793366437Z level=info msg="Executing migration" id="create query_history table v1" 2024-06-18T13:04:00.140 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.794160443Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-06-18T13:04:00.140 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.795016224Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-06-18T13:04:00.140 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.796008277Z level=info msg="Executing migration" id="teams permissions migration" 2024-06-18T13:04:00.140 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.796542851Z level=info msg="Executing migration" id="dashboard permissions" 2024-06-18T13:04:00.140 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.797376076Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-06-18T13:04:00.140 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.798322876Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-06-18T13:04:00.141 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.799093091Z level=info msg="Executing migration" id="alerting notification permissions" 2024-06-18T13:04:00.141 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.799502233Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-06-18T13:04:00.141 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.8004841Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-06-18T13:04:00.141 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.801370472Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-06-18T13:04:00.141 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.802638358Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-06-18T13:04:00.141 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.8029431Z level=info msg="Executing migration" id="create correlation table v1" 2024-06-18T13:04:00.141 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.803946373Z level=info msg="Executing migration" id="add index correlations.uid" 2024-06-18T13:04:00.141 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.804640593Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-06-18T13:04:00.141 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.805479351Z level=info msg="Executing migration" id="add correlation config column" 2024-06-18T13:04:00.141 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.806618004Z level=info msg="Executing migration" id="create entity_events table" 2024-06-18T13:04:00.141 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.807606739Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-06-18T13:04:00.141 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.808319993Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-06-18T13:04:00.141 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.808409943Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-06-18T13:04:00.141 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.809407116Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-06-18T13:04:00.142 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.809598336Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-06-18T13:04:00.142 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.810000023Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-06-18T13:04:00.142 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.810850533Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-06-18T13:04:00.142 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.811688879Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-06-18T13:04:00.142 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.812635014Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-06-18T13:04:00.142 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.813295668Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-06-18T13:04:00.143 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.815975833Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-06-18T13:04:00.143 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.818964205Z level=info msg="Executing migration" id="Drop public config table" 2024-06-18T13:04:00.143 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.821670845Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-06-18T13:04:00.143 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.822114162Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-06-18T13:04:00.143 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.824963535Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-06-18T13:04:00.143 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.827971703Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-06-18T13:04:00.143 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.830696211Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-06-18T13:04:00.143 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.834405506Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-06-18T13:04:00.143 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.837413542Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-06-18T13:04:00.143 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.840277724Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-06-18T13:04:00.143 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.842943328Z level=info msg="Executing migration" id="add share column" 2024-06-18T13:04:00.143 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.84411582Z level=info msg="Executing migration" id="create default alerting folders" 2024-06-18T13:04:00.143 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.847043399Z level=info msg="Executing migration" id="create file table" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.850107745Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.852745758Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.853261749Z level=info msg="Executing migration" id="create file_meta table" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.856053804Z level=info msg="Executing migration" id="file table idx: path key" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.858752978Z level=info msg="Executing migration" id="set path collation in file table" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.86142301Z level=info msg="Executing migration" id="managed permissions migration" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.861903837Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.864604885Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.86735851Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.869917474Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.87023982Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.873058682Z level=info msg="Executing migration" id="update group index for alert rules" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.875660469Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.878401043Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-06-18T13:04:00.144 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.878795301Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.881636335Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.88428837Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.887054025Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.89987785Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.902700718Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.905293352Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.909736189Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.910074288Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.910962097Z level=info msg="Executing migration" id="create folder table" 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.911799217Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.912726688Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.913291995Z level=info msg="Executing migration" id="Update folder title length" 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.914243998Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=migrator t=2024-06-18T13:03:59.91496337Z level=info msg="migrations completed" performed=485 skipped=0 duration=687.019795ms 2024-06-18T13:04:00.145 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=sqlstore t=2024-06-18T13:03:59.915347913Z level=info msg="Created default organization" 2024-06-18T13:04:00.146 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=plugin.finder t=2024-06-18T13:03:59.947203719Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-06-18T13:04:00.146 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=secrets t=2024-06-18T13:03:59.947368363Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-06-18T13:04:00.146 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=query_data t=2024-06-18T13:03:59.951079415Z level=info msg="Query Service initialization" 2024-06-18T13:04:00.146 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:03:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=live.push_http t=2024-06-18T13:03:59.953701998Z level=info msg="Live Push Gateway initialization" 2024-06-18T13:04:00.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:00.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:00 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:00.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:00 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:00.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:00 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:00.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:00 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:00.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:00 smithi167 ceph-mon[37701]: Deploying daemon node-exporter.a on smithi167 2024-06-18T13:04:00.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:00.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:00 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:00.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:00 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:00.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:00 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:00.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:00 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:00.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:00 smithi167 ceph-mon[29496]: Deploying daemon node-exporter.a on smithi167 2024-06-18T13:04:00.552 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:00.552 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:00.552 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:00.552 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:00.552 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:00.552 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-mon[33312]: Deploying daemon node-exporter.a on smithi167 2024-06-18T13:04:00.887 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=infra.usagestats.collector t=2024-06-18T13:04:00.553524619Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-06-18T13:04:00.887 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=provisioning.datasources t=2024-06-18T13:04:00.554798906Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-06-18T13:04:00.887 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=provisioning.datasources t=2024-06-18T13:04:00.565788449Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-06-18T13:04:00.887 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=provisioning.alerting t=2024-06-18T13:04:00.576365099Z level=info msg="starting to provision alerting" 2024-06-18T13:04:00.887 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=provisioning.alerting t=2024-06-18T13:04:00.576375214Z level=info msg="finished to provision alerting" 2024-06-18T13:04:00.887 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=grafanaStorageLogger t=2024-06-18T13:04:00.576567874Z level=info msg="storage starting" 2024-06-18T13:04:00.887 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=ngalert.state.manager t=2024-06-18T13:04:00.576652342Z level=info msg="Warming state cache for startup" 2024-06-18T13:04:00.887 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=http.server t=2024-06-18T13:04:00.587252833Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-06-18T13:04:00.887 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=ngalert.state.manager t=2024-06-18T13:04:00.599146124Z level=info msg="State cache has been initialized" states=0 duration=22.491686ms 2024-06-18T13:04:00.887 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=ticker t=2024-06-18T13:04:00.599200495Z level=info msg=starting first_tick=2024-06-18T13:04:10Z 2024-06-18T13:04:00.887 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=ngalert.multiorg.alertmanager t=2024-06-18T13:04:00.599210562Z level=info msg="starting MultiOrg Alertmanager" 2024-06-18T13:04:00.888 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:04:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=sqlstore.transactions t=2024-06-18T13:04:00.601646604Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-06-18T13:04:00.976 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:00 smithi167 bash[72291]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-06-18T13:04:01.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:01 smithi179 ceph-mon[33312]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:01.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:01 smithi167 ceph-mon[37701]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:01 smithi167 ceph-mon[29496]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:02.476 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 bash[72291]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-06-18T13:04:02.476 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 bash[72291]: Writing manifest to image destination 2024-06-18T13:04:02.877 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 podman[72291]: 2024-06-18 13:04:02.54427929 +0000 UTC m=+2.400712288 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-06-18T13:04:02.878 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 podman[72291]: 2024-06-18 13:04:02.617578194 +0000 UTC m=+2.474011224 container create 10ac283a363106c49ac0662bce4715e2d296e5d5658f2a3bdf3f30cb34400d10 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:03.227 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 podman[72291]: 2024-06-18 13:04:02.878368054 +0000 UTC m=+2.734801051 container init 10ac283a363106c49ac0662bce4715e2d296e5d5658f2a3bdf3f30cb34400d10 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:03.227 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 podman[72291]: 2024-06-18 13:04:02.881722872 +0000 UTC m=+2.738155860 container start 10ac283a363106c49ac0662bce4715e2d296e5d5658f2a3bdf3f30cb34400d10 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:03.227 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.886Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-06-18T13:04:03.227 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.886Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-06-18T13:04:03.227 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-06-18T13:04:03.227 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-06-18T13:04:03.227 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-06-18T13:04:03.227 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-06-18T13:04:03.227 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-06-18T13:04:03.228 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=arp 2024-06-18T13:04:03.228 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=bcache 2024-06-18T13:04:03.228 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=bonding 2024-06-18T13:04:03.228 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=btrfs 2024-06-18T13:04:03.228 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=conntrack 2024-06-18T13:04:03.228 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=cpu 2024-06-18T13:04:03.228 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-06-18T13:04:03.228 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=diskstats 2024-06-18T13:04:03.229 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=dmi 2024-06-18T13:04:03.229 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=edac 2024-06-18T13:04:03.229 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=entropy 2024-06-18T13:04:03.229 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-06-18T13:04:03.229 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=filefd 2024-06-18T13:04:03.229 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=filesystem 2024-06-18T13:04:03.229 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=hwmon 2024-06-18T13:04:03.229 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=infiniband 2024-06-18T13:04:03.229 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=ipvs 2024-06-18T13:04:03.229 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=loadavg 2024-06-18T13:04:03.229 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=mdadm 2024-06-18T13:04:03.229 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=meminfo 2024-06-18T13:04:03.229 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=netclass 2024-06-18T13:04:03.230 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=netdev 2024-06-18T13:04:03.230 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=netstat 2024-06-18T13:04:03.230 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=nfs 2024-06-18T13:04:03.230 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=nfsd 2024-06-18T13:04:03.230 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=nvme 2024-06-18T13:04:03.230 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=os 2024-06-18T13:04:03.230 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-06-18T13:04:03.230 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=pressure 2024-06-18T13:04:03.230 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=rapl 2024-06-18T13:04:03.230 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=schedstat 2024-06-18T13:04:03.230 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=selinux 2024-06-18T13:04:03.230 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=sockstat 2024-06-18T13:04:03.230 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.887Z caller=node_exporter.go:117 level=info collector=softnet 2024-06-18T13:04:03.231 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.888Z caller=node_exporter.go:117 level=info collector=stat 2024-06-18T13:04:03.231 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.888Z caller=node_exporter.go:117 level=info collector=tapestats 2024-06-18T13:04:03.231 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.888Z caller=node_exporter.go:117 level=info collector=textfile 2024-06-18T13:04:03.231 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.888Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-06-18T13:04:03.231 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.888Z caller=node_exporter.go:117 level=info collector=time 2024-06-18T13:04:03.231 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.888Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-06-18T13:04:03.231 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.888Z caller=node_exporter.go:117 level=info collector=uname 2024-06-18T13:04:03.231 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.888Z caller=node_exporter.go:117 level=info collector=vmstat 2024-06-18T13:04:03.231 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.888Z caller=node_exporter.go:117 level=info collector=xfs 2024-06-18T13:04:03.231 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.888Z caller=node_exporter.go:117 level=info collector=zfs 2024-06-18T13:04:03.231 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.888Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-06-18T13:04:03.231 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a[72339]: ts=2024-06-18T13:04:02.888Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-06-18T13:04:03.231 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 bash[72291]: 10ac283a363106c49ac0662bce4715e2d296e5d5658f2a3bdf3f30cb34400d10 2024-06-18T13:04:03.231 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:04:02 smithi167 systemd[1]: Started Ceph node-exporter.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:04:03.615 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:03 smithi179 ceph-mon[33312]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:03.615 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:03 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:03.615 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:03 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:03.615 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:03 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:03.615 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:03 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:03.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:03 smithi167 ceph-mon[29496]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:03.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:03 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:03.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:03 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:03.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:03 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:03.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:03 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:03 smithi167 ceph-mon[37701]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:03 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:03 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:03 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:03 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:03.886 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:03 smithi179 systemd[1]: Starting Ceph node-exporter.b for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:04:04.278 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:03 smithi179 bash[63592]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-06-18T13:04:04.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:04 smithi179 ceph-mon[33312]: Deploying daemon node-exporter.b on smithi179 2024-06-18T13:04:04.636 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:04 smithi179 bash[63592]: Getting image source signatures 2024-06-18T13:04:04.637 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:04 smithi179 bash[63592]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-06-18T13:04:04.637 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:04 smithi179 bash[63592]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-06-18T13:04:04.637 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:04 smithi179 bash[63592]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-06-18T13:04:04.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:04 smithi167 ceph-mon[37701]: Deploying daemon node-exporter.b on smithi179 2024-06-18T13:04:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:04 smithi167 ceph-mon[29496]: Deploying daemon node-exporter.b on smithi179 2024-06-18T13:04:05.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:05 smithi167 ceph-mon[37701]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:05.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:05 smithi167 ceph-mon[29496]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:04:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:04:05.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:05 smithi179 ceph-mon[33312]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:07.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:07 smithi167 ceph-mon[37701]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:07 smithi167 ceph-mon[29496]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:07.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:07 smithi179 ceph-mon[33312]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:08.636 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:08 smithi179 bash[63592]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-06-18T13:04:08.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-06-18T13:04:08.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-06-18T13:04:08.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2024-06-18T13:04:08.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-06-18T13:04:08.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:04:08.638 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:04:08 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:04:08.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-06-18T13:04:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-06-18T13:04:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2024-06-18T13:04:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-06-18T13:04:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:04:08.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-06-18T13:04:08.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-06-18T13:04:08.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2024-06-18T13:04:08.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-06-18T13:04:08.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:04:08.931 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:08 smithi179 bash[63592]: Writing manifest to image destination 2024-06-18T13:04:09.218 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:08 smithi179 podman[63592]: 2024-06-18 13:04:08.910261324 +0000 UTC m=+5.007202652 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-06-18T13:04:09.218 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 podman[63592]: 2024-06-18 13:04:09.017075678 +0000 UTC m=+5.114017000 container create 8248ccef56c1e12bd30e0caaf657a71bd35216a8d20a2128a25d8d3b5035e891 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-18T13:04:09.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-mon[33312]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:09.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]': finished 2024-06-18T13:04:09.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-06-18T13:04:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]': finished 2024-06-18T13:04:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-06-18T13:04:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-mon[33312]: osdmap e72: 8 total, 8 up, 8 in 2024-06-18T13:04:09.637 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 podman[63592]: 2024-06-18 13:04:09.219030998 +0000 UTC m=+5.315972318 container init 8248ccef56c1e12bd30e0caaf657a71bd35216a8d20a2128a25d8d3b5035e891 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-18T13:04:09.637 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 podman[63592]: 2024-06-18 13:04:09.222444909 +0000 UTC m=+5.319386229 container start 8248ccef56c1e12bd30e0caaf657a71bd35216a8d20a2128a25d8d3b5035e891 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-18T13:04:09.637 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.226Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-06-18T13:04:09.637 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.226Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-06-18T13:04:09.637 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.226Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-06-18T13:04:09.637 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.226Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-06-18T13:04:09.638 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-06-18T13:04:09.638 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-06-18T13:04:09.638 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-06-18T13:04:09.638 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=arp 2024-06-18T13:04:09.638 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=bcache 2024-06-18T13:04:09.638 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=bonding 2024-06-18T13:04:09.638 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=btrfs 2024-06-18T13:04:09.638 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=conntrack 2024-06-18T13:04:09.638 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=cpu 2024-06-18T13:04:09.638 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-06-18T13:04:09.639 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=diskstats 2024-06-18T13:04:09.639 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=dmi 2024-06-18T13:04:09.639 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=edac 2024-06-18T13:04:09.639 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=entropy 2024-06-18T13:04:09.639 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-06-18T13:04:09.639 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=filefd 2024-06-18T13:04:09.639 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=filesystem 2024-06-18T13:04:09.639 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=hwmon 2024-06-18T13:04:09.639 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=infiniband 2024-06-18T13:04:09.639 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=ipvs 2024-06-18T13:04:09.640 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=loadavg 2024-06-18T13:04:09.640 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=mdadm 2024-06-18T13:04:09.640 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=meminfo 2024-06-18T13:04:09.640 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=netclass 2024-06-18T13:04:09.640 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=netdev 2024-06-18T13:04:09.640 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=netstat 2024-06-18T13:04:09.640 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=nfs 2024-06-18T13:04:09.640 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=nfsd 2024-06-18T13:04:09.640 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=nvme 2024-06-18T13:04:09.640 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=os 2024-06-18T13:04:09.640 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-06-18T13:04:09.640 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=pressure 2024-06-18T13:04:09.640 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=rapl 2024-06-18T13:04:09.640 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=schedstat 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=selinux 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=sockstat 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=softnet 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=stat 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=tapestats 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=textfile 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=time 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=uname 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=vmstat 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=xfs 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.227Z caller=node_exporter.go:117 level=info collector=zfs 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.228Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-06-18T13:04:09.641 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b[63642]: ts=2024-06-18T13:04:09.228Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-06-18T13:04:09.642 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 bash[63592]: 8248ccef56c1e12bd30e0caaf657a71bd35216a8d20a2128a25d8d3b5035e891 2024-06-18T13:04:09.642 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:04:09 smithi179 systemd[1]: Started Ceph node-exporter.b for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:04:09.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:09 smithi167 ceph-mon[37701]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:09.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]': finished 2024-06-18T13:04:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-06-18T13:04:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]': finished 2024-06-18T13:04:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-06-18T13:04:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:09 smithi167 ceph-mon[37701]: osdmap e72: 8 total, 8 up, 8 in 2024-06-18T13:04:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:09 smithi167 ceph-mon[29496]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]': finished 2024-06-18T13:04:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-06-18T13:04:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]': finished 2024-06-18T13:04:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-06-18T13:04:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:09 smithi167 ceph-mon[29496]: osdmap e72: 8 total, 8 up, 8 in 2024-06-18T13:04:10.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:10.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[29496]: osdmap e73: 8 total, 8 up, 8 in 2024-06-18T13:04:10.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:04:10.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:10.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[37701]: osdmap e73: 8 total, 8 up, 8 in 2024-06-18T13:04:10.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:04:10.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:10 smithi179 ceph-mon[33312]: osdmap e73: 8 total, 8 up, 8 in 2024-06-18T13:04:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:04:11.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:11 smithi179 ceph-mon[33312]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:11.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:11 smithi179 ceph-mon[33312]: Health check failed: Reduced data availability: 1 pg inactive, 4 pgs peering (PG_AVAILABILITY) 2024-06-18T13:04:11.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:11 smithi167 ceph-mon[37701]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:11.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:11 smithi167 ceph-mon[37701]: Health check failed: Reduced data availability: 1 pg inactive, 4 pgs peering (PG_AVAILABILITY) 2024-06-18T13:04:11.715 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:11 smithi167 ceph-mon[29496]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:11.715 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:11 smithi167 ceph-mon[29496]: Health check failed: Reduced data availability: 1 pg inactive, 4 pgs peering (PG_AVAILABILITY) 2024-06-18T13:04:13.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:13 smithi179 ceph-mon[33312]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:04:13.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:13 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:13.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:13 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:13 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:13.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:13 smithi167 ceph-mon[37701]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:04:13.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:13 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:13.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:13 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:13.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:13 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:13.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:13 smithi167 ceph-mon[29496]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:04:13.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:13 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:13.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:13 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:13.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:13 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:14.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[37701]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-06-18T13:04:14.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:14.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:14.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:04:14.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:04:14.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:14.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[37701]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-18T13:04:14.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[37701]: Reconfiguring daemon alertmanager.a on smithi167 2024-06-18T13:04:14.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[37701]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 4 pgs peering) 2024-06-18T13:04:14.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[37701]: Cluster is now healthy 2024-06-18T13:04:14.573 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[29496]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-06-18T13:04:14.573 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:14.573 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:14.573 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:04:14.573 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:04:14.573 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:14.573 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[29496]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-18T13:04:14.573 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[29496]: Reconfiguring daemon alertmanager.a on smithi167 2024-06-18T13:04:14.573 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[29496]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 4 pgs peering) 2024-06-18T13:04:14.574 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:14 smithi167 ceph-mon[29496]: Cluster is now healthy 2024-06-18T13:04:14.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:14 smithi179 ceph-mon[33312]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-06-18T13:04:14.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:14 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:14 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:14 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:04:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:14 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:04:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:14 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:14 smithi179 ceph-mon[33312]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-18T13:04:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:14 smithi179 ceph-mon[33312]: Reconfiguring daemon alertmanager.a on smithi167 2024-06-18T13:04:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:14 smithi179 ceph-mon[33312]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 4 pgs peering) 2024-06-18T13:04:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:14 smithi179 ceph-mon[33312]: Cluster is now healthy 2024-06-18T13:04:15.363 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:04:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:04:16.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:16 smithi167 ceph-mon[29496]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-06-18T13:04:16.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:16 smithi167 ceph-mon[37701]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-06-18T13:04:16.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:16 smithi179 ceph-mon[33312]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-06-18T13:04:17.675 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:17 smithi167 systemd[1]: Stopping Ceph alertmanager.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:04:17.976 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:17 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[71439]: ts=2024-06-18T13:04:17.676Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-06-18T13:04:17.976 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:17 smithi167 podman[73055]: 2024-06-18 13:04:17.69676811 +0000 UTC m=+0.152172875 container died 44dbeec8b46cd8bcd5b8045d108218f2949b010b6c47928873954e444dc41fa2 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:18.377 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:18 smithi167 podman[73055]: 2024-06-18 13:04:18.085409623 +0000 UTC m=+0.540814388 container cleanup 44dbeec8b46cd8bcd5b8045d108218f2949b010b6c47928873954e444dc41fa2 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:18.377 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:18 smithi167 bash[73055]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a 2024-06-18T13:04:18.679 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:18 smithi167 ceph-mon[37701]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 24 B/s, 1 objects/s recovering 2024-06-18T13:04:18.680 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:18 smithi167 podman[73069]: 2024-06-18 13:04:18.378653991 +0000 UTC m=+0.681323663 container remove 44dbeec8b46cd8bcd5b8045d108218f2949b010b6c47928873954e444dc41fa2 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:18.680 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:18 smithi167 podman[73069]: 2024-06-18 13:04:18.479909441 +0000 UTC m=+0.782579127 volume remove e78f68282bc03148535e7376ed2bc479de702c4f5293f8635a54e2a25b7174a2 2024-06-18T13:04:18.680 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:18 smithi167 ceph-mon[29496]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 24 B/s, 1 objects/s recovering 2024-06-18T13:04:18.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:18 smithi179 ceph-mon[33312]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 24 B/s, 1 objects/s recovering 2024-06-18T13:04:18.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:04:18 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:04:18.976 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:18 smithi167 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-06-18T13:04:18.976 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:18 smithi167 systemd[1]: Stopped Ceph alertmanager.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:04:18.976 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:18 smithi167 systemd[1]: Starting Ceph alertmanager.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:04:18.976 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:18 smithi167 podman[73160]: 2024-06-18 13:04:18.821310199 +0000 UTC m=+0.020800884 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-06-18T13:04:18.976 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:18 smithi167 podman[73160]: 2024-06-18 13:04:18.930970259 +0000 UTC m=+0.130460933 volume create 08e6397bddcbbbbb8301af78c36d11f3f7016e1b7380682f685beb94b9211c7e 2024-06-18T13:04:19.359 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:19 smithi167 podman[73160]: 2024-06-18 13:04:19.040135046 +0000 UTC m=+0.239625716 container create 60a8f2c52c41af9d152f7aea0b6000d6e8d6db741793aab0d992e015bbbbef52 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:19.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:19 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:19 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:19.727 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:19 smithi167 podman[73160]: 2024-06-18 13:04:19.360144041 +0000 UTC m=+0.559634721 container init 60a8f2c52c41af9d152f7aea0b6000d6e8d6db741793aab0d992e015bbbbef52 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:19.727 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:19 smithi167 podman[73160]: 2024-06-18 13:04:19.363292451 +0000 UTC m=+0.562783120 container start 60a8f2c52c41af9d152f7aea0b6000d6e8d6db741793aab0d992e015bbbbef52 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:19.727 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:19 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[73175]: ts=2024-06-18T13:04:19.378Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-06-18T13:04:19.727 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:19 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[73175]: ts=2024-06-18T13:04:19.378Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-06-18T13:04:19.727 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:19 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[73175]: ts=2024-06-18T13:04:19.379Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.167 port=9094 2024-06-18T13:04:19.727 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:19 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[73175]: ts=2024-06-18T13:04:19.380Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-06-18T13:04:19.727 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:19 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[73175]: ts=2024-06-18T13:04:19.409Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-18T13:04:19.727 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:19 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[73175]: ts=2024-06-18T13:04:19.410Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-18T13:04:19.727 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:19 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[73175]: ts=2024-06-18T13:04:19.413Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-06-18T13:04:19.728 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:19 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[73175]: ts=2024-06-18T13:04:19.413Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-06-18T13:04:19.728 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:19 smithi167 bash[73160]: 60a8f2c52c41af9d152f7aea0b6000d6e8d6db741793aab0d992e015bbbbef52 2024-06-18T13:04:19.728 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:19 smithi167 systemd[1]: Started Ceph alertmanager.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:04:19.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:19 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:20.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:20 smithi179 ceph-mon[33312]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1008 B/s rd, 0 op/s; 21 B/s, 1 objects/s recovering 2024-06-18T13:04:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:20 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:20 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:20 smithi179 ceph-mon[33312]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-18T13:04:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:20 smithi179 ceph-mon[33312]: Reconfiguring daemon prometheus.a on smithi179 2024-06-18T13:04:20.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:20 smithi167 ceph-mon[37701]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1008 B/s rd, 0 op/s; 21 B/s, 1 objects/s recovering 2024-06-18T13:04:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:20 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:20 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:20 smithi167 ceph-mon[37701]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-18T13:04:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:20 smithi167 ceph-mon[37701]: Reconfiguring daemon prometheus.a on smithi179 2024-06-18T13:04:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:20 smithi167 ceph-mon[29496]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1008 B/s rd, 0 op/s; 21 B/s, 1 objects/s recovering 2024-06-18T13:04:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:20 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:20 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:20 smithi167 ceph-mon[29496]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-18T13:04:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:20 smithi167 ceph-mon[29496]: Reconfiguring daemon prometheus.a on smithi179 2024-06-18T13:04:21.727 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:21 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[73175]: ts=2024-06-18T13:04:21.380Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.00001441s 2024-06-18T13:04:22.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:22 smithi179 ceph-mon[33312]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 1 objects/s recovering 2024-06-18T13:04:22.887 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:22 smithi179 systemd[1]: Stopping Ceph prometheus.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:04:22.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:22 smithi167 ceph-mon[37701]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 1 objects/s recovering 2024-06-18T13:04:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:22 smithi167 ceph-mon[29496]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 1 objects/s recovering 2024-06-18T13:04:23.154 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[61411]: ts=2024-06-18T13:04:22.887Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-06-18T13:04:23.154 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[61411]: ts=2024-06-18T13:04:22.888Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-06-18T13:04:23.154 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[61411]: ts=2024-06-18T13:04:22.888Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-06-18T13:04:23.154 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[61411]: ts=2024-06-18T13:04:22.888Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-06-18T13:04:23.154 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[61411]: ts=2024-06-18T13:04:22.888Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-06-18T13:04:23.154 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[61411]: ts=2024-06-18T13:04:22.888Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-06-18T13:04:23.154 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[61411]: ts=2024-06-18T13:04:22.888Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-06-18T13:04:23.154 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[61411]: ts=2024-06-18T13:04:22.888Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-06-18T13:04:23.154 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[61411]: ts=2024-06-18T13:04:22.888Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-06-18T13:04:23.155 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[61411]: ts=2024-06-18T13:04:22.888Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-06-18T13:04:23.155 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[61411]: ts=2024-06-18T13:04:22.888Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-06-18T13:04:23.155 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[61411]: ts=2024-06-18T13:04:22.888Z caller=main.go:1147 level=info msg="See you next time!" 2024-06-18T13:04:23.155 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:22 smithi179 podman[64537]: 2024-06-18 13:04:22.897916258 +0000 UTC m=+0.100644180 container died a0f00228eaaf800ffc209e9dabc817571765bf966d0e8dff4fccf9a31cfd64a3 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:23.499 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:23 smithi179 podman[64537]: 2024-06-18 13:04:23.254013248 +0000 UTC m=+0.456741172 container cleanup a0f00228eaaf800ffc209e9dabc817571765bf966d0e8dff4fccf9a31cfd64a3 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:23.499 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:23 smithi179 bash[64537]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a 2024-06-18T13:04:23.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:23 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:04:23.887 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:23 smithi179 podman[64548]: 2024-06-18 13:04:23.563578357 +0000 UTC m=+0.666881675 container remove a0f00228eaaf800ffc209e9dabc817571765bf966d0e8dff4fccf9a31cfd64a3 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:23.887 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:23 smithi179 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-06-18T13:04:23.887 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:23 smithi179 systemd[1]: Stopped Ceph prometheus.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:04:23.887 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:23 smithi179 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@prometheus.a.service: Consumed 1.108s CPU time. 2024-06-18T13:04:23.887 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:23 smithi179 systemd[1]: Starting Ceph prometheus.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:04:23.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:23 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:04:23.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:23 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:04:24.217 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:23 smithi179 podman[64641]: 2024-06-18 13:04:23.857235108 +0000 UTC m=+0.018453361 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-06-18T13:04:24.217 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:23 smithi179 podman[64641]: 2024-06-18 13:04:23.989708302 +0000 UTC m=+0.150926592 container create 4d7013e6be9e26f8d87e0546fed72dad06f16b2176aa3971271593cb92bd05b0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:24.217 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 podman[64641]: 2024-06-18 13:04:24.192205616 +0000 UTC m=+0.353423870 container init 4d7013e6be9e26f8d87e0546fed72dad06f16b2176aa3971271593cb92bd05b0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:24.217 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 podman[64641]: 2024-06-18 13:04:24.194938579 +0000 UTC m=+0.356156824 container start 4d7013e6be9e26f8d87e0546fed72dad06f16b2176aa3971271593cb92bd05b0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-18T13:04:24.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:24 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:24] ENGINE Bus STOPPING 2024-06-18T13:04:24.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:24 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:24] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-18T13:04:24.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:24 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:24] ENGINE Bus STOPPED 2024-06-18T13:04:24.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:24 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:24] ENGINE Bus STARTING 2024-06-18T13:04:24.637 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.218Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-06-18T13:04:24.637 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.218Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-06-18T13:04:24.637 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.218Z caller=main.go:570 level=info host_details="(Linux 5.14.0-460.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Jun 6 12:49:42 UTC 2024 x86_64 smithi179 (none))" 2024-06-18T13:04:24.637 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.218Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-06-18T13:04:24.637 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.218Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-06-18T13:04:24.637 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.221Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-06-18T13:04:24.637 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.221Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-06-18T13:04:24.637 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.222Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-06-18T13:04:24.637 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.222Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-06-18T13:04:24.637 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.223Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-06-18T13:04:24.637 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.223Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.77µs 2024-06-18T13:04:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.223Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-06-18T13:04:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.223Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-06-18T13:04:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.224Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-06-18T13:04:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.224Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=28.166µs wal_replay_duration=1.229459ms wbl_replay_duration=149ns total_replay_duration=1.281038ms 2024-06-18T13:04:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.225Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-06-18T13:04:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.225Z caller=main.go:1029 level=info msg="TSDB started" 2024-06-18T13:04:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.225Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-06-18T13:04:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.246Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.164149ms db_storage=867ns remote_storage=1.071µs web_handler=390ns query_engine=547ns scrape=376.381µs scrape_sd=83.348µs notify=9.476µs notify_sd=8.345µs rules=19.396199ms tracing=5.889µs 2024-06-18T13:04:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.246Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-06-18T13:04:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:04:24.246Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-06-18T13:04:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 bash[64641]: 4d7013e6be9e26f8d87e0546fed72dad06f16b2176aa3971271593cb92bd05b0 2024-06-18T13:04:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:04:24 smithi179 systemd[1]: Started Ceph prometheus.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:04:24.867 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:24 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:24] ENGINE Serving on http://:::9283 2024-06-18T13:04:24.867 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:24 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:24] ENGINE Bus STARTED 2024-06-18T13:04:24.867 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:24 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:24] ENGINE Bus STOPPING 2024-06-18T13:04:25.226 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:25] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-18T13:04:25.227 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:25] ENGINE Bus STOPPED 2024-06-18T13:04:25.227 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:25] ENGINE Bus STARTING 2024-06-18T13:04:25.227 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:25] ENGINE Serving on http://:::9283 2024-06-18T13:04:25.227 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:25] ENGINE Bus STARTED 2024-06-18T13:04:25.227 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:25] ENGINE Bus STOPPING 2024-06-18T13:04:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 18 B/s, 1 objects/s recovering 2024-06-18T13:04:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-18T13:04:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-18T13:04:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi167.front.sepia.ceph.com:9093"}]: dispatch 2024-06-18T13:04:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi167.front.sepia.ceph.com:9093"}]: dispatch 2024-06-18T13:04:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-18T13:04:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-18T13:04:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi179.front.sepia.ceph.com:9095"}]: dispatch 2024-06-18T13:04:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi179.front.sepia.ceph.com:9095"}]: dispatch 2024-06-18T13:04:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-18T13:04:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-18T13:04:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi179.front.sepia.ceph.com:3000"}]: dispatch 2024-06-18T13:04:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi179.front.sepia.ceph.com:3000"}]: dispatch 2024-06-18T13:04:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:04:25.608 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 18 B/s, 1 objects/s recovering 2024-06-18T13:04:25.608 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.608 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi167.front.sepia.ceph.com:9093"}]: dispatch 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi167.front.sepia.ceph.com:9093"}]: dispatch 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi179.front.sepia.ceph.com:9095"}]: dispatch 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi179.front.sepia.ceph.com:9095"}]: dispatch 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi179.front.sepia.ceph.com:3000"}]: dispatch 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi179.front.sepia.ceph.com:3000"}]: dispatch 2024-06-18T13:04:25.609 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.610 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:04:25.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 18 B/s, 1 objects/s recovering 2024-06-18T13:04:25.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-18T13:04:25.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-18T13:04:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi167.front.sepia.ceph.com:9093"}]: dispatch 2024-06-18T13:04:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi167.front.sepia.ceph.com:9093"}]: dispatch 2024-06-18T13:04:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-18T13:04:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-18T13:04:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi179.front.sepia.ceph.com:9095"}]: dispatch 2024-06-18T13:04:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi179.front.sepia.ceph.com:9095"}]: dispatch 2024-06-18T13:04:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-18T13:04:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-18T13:04:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi179.front.sepia.ceph.com:3000"}]: dispatch 2024-06-18T13:04:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi179.front.sepia.ceph.com:3000"}]: dispatch 2024-06-18T13:04:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:25.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:04:25.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:25] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-18T13:04:25.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:25] ENGINE Bus STOPPED 2024-06-18T13:04:25.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:25] ENGINE Bus STARTING 2024-06-18T13:04:25.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:25] ENGINE Serving on http://:::9283 2024-06-18T13:04:25.977 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:04:25] ENGINE Bus STARTED 2024-06-18T13:04:27.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:27 smithi179 ceph-mon[33312]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:27.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:27 smithi167 ceph-mon[29496]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:27 smithi167 ceph-mon[37701]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:28.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:04:28 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:04:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:29 smithi179 ceph-mon[33312]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:29 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:29 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:29 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:29 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:29 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:04:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:29 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:04:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:29 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:29.726 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[73175]: ts=2024-06-18T13:04:29.384Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003199525s 2024-06-18T13:04:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[37701]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:04:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:04:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[29496]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:04:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:04:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:29 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:04:30.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:30.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:30.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:31.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:31 smithi179 ceph-mon[33312]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:31.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:31 smithi167 ceph-mon[37701]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:31 smithi167 ceph-mon[29496]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:32.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:32 smithi179 ceph-mon[33312]: pgmap v47: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:32.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:32 smithi167 ceph-mon[37701]: pgmap v47: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:32 smithi167 ceph-mon[29496]: pgmap v47: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:34.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:34 smithi179 ceph-mon[33312]: pgmap v48: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:34.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:34 smithi167 ceph-mon[37701]: pgmap v48: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:34.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:34 smithi167 ceph-mon[29496]: pgmap v48: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:35.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:04:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:04:36.831 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:36 smithi179 ceph-mon[33312]: pgmap v49: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:36.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:36 smithi167 ceph-mon[29496]: pgmap v49: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:36.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:36 smithi167 ceph-mon[37701]: pgmap v49: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:38.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:38 smithi179 ceph-mon[33312]: pgmap v50: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:38.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:38 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:04:38.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:04:38 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:04:38.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:38 smithi167 ceph-mon[37701]: pgmap v50: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:38.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:38 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:04:38.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:38 smithi167 ceph-mon[29496]: pgmap v50: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:38 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:04:39.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:39 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:39.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:39 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:39.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:39 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:40 smithi179 ceph-mon[33312]: pgmap v51: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:40.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:40 smithi167 ceph-mon[37701]: pgmap v51: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:40 smithi167 ceph-mon[29496]: pgmap v51: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:42.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:42 smithi179 ceph-mon[33312]: pgmap v52: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:42.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:42 smithi167 ceph-mon[37701]: pgmap v52: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:42.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:42 smithi167 ceph-mon[29496]: pgmap v52: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:44.778 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:44 smithi167 ceph-mon[37701]: pgmap v53: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:44.779 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:44 smithi167 ceph-mon[29496]: pgmap v53: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:44 smithi179 ceph-mon[33312]: pgmap v53: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:45.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:04:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:04:46.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:46 smithi179 ceph-mon[33312]: pgmap v54: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:46.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:46 smithi167 ceph-mon[29496]: pgmap v54: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:46 smithi167 ceph-mon[37701]: pgmap v54: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:48.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:48 smithi179 ceph-mon[33312]: pgmap v55: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:48.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:04:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:04:48.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:48 smithi167 ceph-mon[29496]: pgmap v55: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:48 smithi167 ceph-mon[37701]: pgmap v55: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:49.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:49 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:49.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:49 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:49.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:49 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:04:50.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:50 smithi179 ceph-mon[33312]: pgmap v56: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:50.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:50 smithi167 ceph-mon[37701]: pgmap v56: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:50.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:50 smithi167 ceph-mon[29496]: pgmap v56: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:52.863 INFO:tasks.workunit.client.0.smithi167.stderr:Note: switching to 'f18f3f1cbd11e4a41c557e616d172f24deb91473'. 2024-06-18T13:04:52.864 INFO:tasks.workunit.client.0.smithi167.stderr: 2024-06-18T13:04:52.864 INFO:tasks.workunit.client.0.smithi167.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-18T13:04:52.864 INFO:tasks.workunit.client.0.smithi167.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-18T13:04:52.864 INFO:tasks.workunit.client.0.smithi167.stderr:state without impacting any branches by switching back to a branch. 2024-06-18T13:04:52.864 INFO:tasks.workunit.client.0.smithi167.stderr: 2024-06-18T13:04:52.864 INFO:tasks.workunit.client.0.smithi167.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-18T13:04:52.864 INFO:tasks.workunit.client.0.smithi167.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-18T13:04:52.864 INFO:tasks.workunit.client.0.smithi167.stderr: 2024-06-18T13:04:52.864 INFO:tasks.workunit.client.0.smithi167.stderr: git switch -c 2024-06-18T13:04:52.864 INFO:tasks.workunit.client.0.smithi167.stderr: 2024-06-18T13:04:52.864 INFO:tasks.workunit.client.0.smithi167.stderr:Or undo this operation with: 2024-06-18T13:04:52.865 INFO:tasks.workunit.client.0.smithi167.stderr: 2024-06-18T13:04:52.865 INFO:tasks.workunit.client.0.smithi167.stderr: git switch - 2024-06-18T13:04:52.865 INFO:tasks.workunit.client.0.smithi167.stderr: 2024-06-18T13:04:52.865 INFO:tasks.workunit.client.0.smithi167.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-18T13:04:52.865 INFO:tasks.workunit.client.0.smithi167.stderr: 2024-06-18T13:04:52.865 INFO:tasks.workunit.client.0.smithi167.stderr:HEAD is now at f18f3f1cbd1 Merge pull request #58034 from Matan-B/wip-57593-squid 2024-06-18T13:04:52.868 DEBUG:teuthology.orchestra.run.smithi167:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-18T13:04:52.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:52 smithi179 ceph-mon[33312]: pgmap v57: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:52.906 INFO:tasks.workunit.client.0.smithi167.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-18T13:04:52.908 INFO:tasks.workunit.client.0.smithi167.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-18T13:04:52.908 INFO:tasks.workunit.client.0.smithi167.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-18T13:04:52.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:52 smithi167 ceph-mon[29496]: pgmap v57: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:52.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:52 smithi167 ceph-mon[37701]: pgmap v57: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:04:53.719 INFO:tasks.workunit.client.0.smithi167.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-18T13:04:53.784 INFO:tasks.workunit.client.0.smithi167.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-18T13:04:53.802 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:53 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:04:53.825 INFO:tasks.workunit.client.0.smithi167.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-18T13:04:53.827 INFO:tasks.workunit.client.0.smithi167.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-18T13:04:53.827 INFO:tasks.workunit.client.0.smithi167.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-18T13:04:53.866 INFO:tasks.workunit.client.0.smithi167.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-18T13:04:53.870 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-18T13:04:53.870 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-18T13:04:53.927 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-06-18T13:04:53.928 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-06-18T13:04:53.928 DEBUG:teuthology.orchestra.run.smithi167:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f18f3f1cbd11e4a41c557e616d172f24deb91473 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-06-18T13:04:53.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:53 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:04:53.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:53 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:04:53.995 INFO:tasks.workunit.client.0.smithi167.stderr:+ parallel=1 2024-06-18T13:04:53.995 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' '' = --serial ']' 2024-06-18T13:04:53.995 INFO:tasks.workunit.client.0.smithi167.stderr:+ crimson=0 2024-06-18T13:04:53.996 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' '' = --crimson ']' 2024-06-18T13:04:53.996 INFO:tasks.workunit.client.0.smithi167.stderr:+ color= 2024-06-18T13:04:53.996 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -t 1 ']' 2024-06-18T13:04:53.996 INFO:tasks.workunit.client.0.smithi167.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-06-18T13:04:53.996 INFO:tasks.workunit.client.0.smithi167.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-06-18T13:04:53.996 INFO:tasks.workunit.client.0.smithi167.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-06-18T13:04:53.997 INFO:tasks.workunit.client.0.smithi167.stderr:+ declare -A pids 2024-06-18T13:04:53.997 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:53.998 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:53.998 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_aio 2024-06-18T13:04:53.998 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_aio' 2024-06-18T13:04:53.998 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_aio 2024-06-18T13:04:53.998 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:53.999 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_aio 2024-06-18T13:04:53.999 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=73905 2024-06-18T13:04:53.999 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_aio on pid 73905' 2024-06-18T13:04:53.999 INFO:tasks.workunit.client.0.smithi167.stdout:test api_aio on pid 73905 2024-06-18T13:04:54.003 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=73905 2024-06-18T13:04:54.003 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-06-18T13:04:54.003 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.003 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.003 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_aio_pp 2024-06-18T13:04:54.003 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_aio_pp' 2024-06-18T13:04:54.003 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_aio_pp 2024-06-18T13:04:54.003 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.004 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.004 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.004 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.004 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.004 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.004 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.004 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.004 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.004 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.004 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.004 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.005 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.005 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.005 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.005 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.005 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.005 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.005 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.005 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.005 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.005 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.006 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.006 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.006 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.006 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.006 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_aio_pp 2024-06-18T13:04:54.006 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=73912 2024-06-18T13:04:54.006 INFO:tasks.workunit.client.0.smithi167.stdout:test api_aio_pp on pid 73912 2024-06-18T13:04:54.011 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_aio_pp on pid 73912' 2024-06-18T13:04:54.011 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=73912 2024-06-18T13:04:54.011 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.011 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.011 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-06-18T13:04:54.012 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_io 2024-06-18T13:04:54.012 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_io' 2024-06-18T13:04:54.012 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_io 2024-06-18T13:04:54.012 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.012 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.012 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.012 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.012 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.012 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.012 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.013 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.013 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.013 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.013 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.013 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.013 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.013 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.013 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.013 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.013 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.014 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.014 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.014 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.014 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.014 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.014 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.014 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.014 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.014 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.014 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.014 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.015 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.015 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.015 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.015 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.015 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.015 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.015 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.015 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.015 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.015 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.015 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.015 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.016 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.016 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.016 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.016 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.016 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.016 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.016 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_io 2024-06-18T13:04:54.016 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=73921 2024-06-18T13:04:54.016 INFO:tasks.workunit.client.0.smithi167.stdout:test api_io on pid 73921 2024-06-18T13:04:54.022 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_io on pid 73921' 2024-06-18T13:04:54.022 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=73921 2024-06-18T13:04:54.022 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.022 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.022 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-06-18T13:04:54.022 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_io_pp 2024-06-18T13:04:54.023 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_io_pp' 2024-06-18T13:04:54.023 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_io_pp 2024-06-18T13:04:54.023 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.023 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.023 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.023 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.023 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.023 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.023 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.023 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.023 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.023 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.024 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.024 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.024 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.024 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.024 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.024 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.024 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.024 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.024 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.024 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.024 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.024 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.025 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.025 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.025 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.025 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.025 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.025 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.025 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.025 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.025 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.025 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.025 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.026 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.026 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.026 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.026 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.026 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.026 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.026 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.026 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.026 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.026 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.026 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.027 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.027 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.027 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.027 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.027 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.027 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.027 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.027 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.027 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.027 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.027 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.027 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_io_pp 2024-06-18T13:04:54.027 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=73932 2024-06-18T13:04:54.028 INFO:tasks.workunit.client.0.smithi167.stdout:test api_io_pp on pid 73932 2024-06-18T13:04:54.033 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_io_pp on pid 73932' 2024-06-18T13:04:54.033 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=73932 2024-06-18T13:04:54.033 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.033 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.033 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-06-18T13:04:54.033 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_asio 2024-06-18T13:04:54.034 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_asio' 2024-06-18T13:04:54.034 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_asio 2024-06-18T13:04:54.034 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.034 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.034 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.034 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.034 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.034 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.034 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.034 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.034 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.034 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.035 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.035 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.035 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.035 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.035 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.035 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.035 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.035 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.035 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.035 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.035 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.035 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.036 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.036 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.036 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.036 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.036 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.036 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.036 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.036 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.036 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.036 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.036 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.037 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.037 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.037 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.037 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.037 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.037 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.037 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.037 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.037 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.037 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.037 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.037 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.038 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.038 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.038 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.038 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.038 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.038 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.038 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.038 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.038 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.038 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.038 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.039 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.039 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.039 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.039 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.039 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.039 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.039 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.039 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.039 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.039 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.039 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.039 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.039 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.040 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.040 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.040 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.040 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.040 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_asio 2024-06-18T13:04:54.040 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=73948 2024-06-18T13:04:54.040 INFO:tasks.workunit.client.0.smithi167.stdout:test api_asio on pid 73948 2024-06-18T13:04:54.047 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_asio on pid 73948' 2024-06-18T13:04:54.047 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=73948 2024-06-18T13:04:54.047 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.047 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.047 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-06-18T13:04:54.047 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.048 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.048 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_list 2024-06-18T13:04:54.048 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.048 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.048 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.048 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_list' 2024-06-18T13:04:54.048 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.048 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.048 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.048 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.048 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.049 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.049 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.049 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.049 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.049 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.049 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_list 2024-06-18T13:04:54.049 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.049 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.049 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.049 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.049 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.049 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.049 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.049 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.050 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.050 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.050 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.050 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.050 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.050 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.050 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.050 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.050 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.050 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.051 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.051 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.051 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.051 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.051 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.051 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.051 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.051 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.051 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.051 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.051 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.052 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.052 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.052 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.052 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.052 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.052 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.052 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.052 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.052 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.052 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.052 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.052 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.053 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.053 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.053 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.053 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.053 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.053 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.053 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.053 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.053 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.053 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.053 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.053 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.053 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.053 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.054 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.054 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.054 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.054 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.054 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.054 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.054 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.054 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.054 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.054 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.054 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.054 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.054 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.055 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.055 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.055 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.055 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.055 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.055 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.055 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.055 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.055 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.055 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.055 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.055 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.055 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.056 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.056 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.056 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.056 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.056 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/73905/exe 2024-06-18T13:04:54.056 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_list 2024-06-18T13:04:54.056 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.056 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.056 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=73965 2024-06-18T13:04:54.056 INFO:tasks.workunit.client.0.smithi167.stdout:test api_list on pid 73965 2024-06-18T13:04:54.062 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.062 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_list on pid 73965' 2024-06-18T13:04:54.062 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.062 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=73965 2024-06-18T13:04:54.062 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.062 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.062 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.063 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-06-18T13:04:54.063 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.063 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.063 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_lock 2024-06-18T13:04:54.063 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.063 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.063 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.063 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.063 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.063 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.063 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.063 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.064 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.064 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.064 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.064 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.064 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_lock' 2024-06-18T13:04:54.064 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.064 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.064 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.064 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.064 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.064 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.064 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.064 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.065 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.065 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.065 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.065 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.065 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.065 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.065 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.065 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.065 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_lock 2024-06-18T13:04:54.065 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.065 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.065 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.065 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.065 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.066 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.066 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.066 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.066 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.066 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.066 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.066 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.066 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.066 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.066 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.066 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.066 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.067 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.067 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.067 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.067 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.067 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.067 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.067 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.067 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.067 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.067 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.067 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.067 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.067 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.068 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.068 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.068 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.068 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.068 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2024-06-18T13:04:54.068 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.068 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.068 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.068 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.068 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.068 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.068 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.068 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.069 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.069 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_aio.log 2024-06-18T13:04:54.069 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.069 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.069 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.069 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.069 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.069 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.069 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.069 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.069 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.069 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.070 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.070 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.070 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.070 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.070 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.070 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.070 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.070 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.070 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.070 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.070 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.070 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.070 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.071 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.071 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.071 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.071 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.071 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.071 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.071 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.071 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.071 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.071 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.071 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.071 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.071 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.071 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.072 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.072 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/73912/exe 2024-06-18T13:04:54.072 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_aio: /' 2024-06-18T13:04:54.072 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.072 INFO:tasks.workunit.client.0.smithi167.stdout:test api_lock on pid 73985 2024-06-18T13:04:54.074 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.074 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_lock 2024-06-18T13:04:54.075 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=73985 2024-06-18T13:04:54.075 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_lock on pid 73985' 2024-06-18T13:04:54.075 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=73985 2024-06-18T13:04:54.075 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.075 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.075 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-06-18T13:04:54.075 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_lock_pp 2024-06-18T13:04:54.075 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.075 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.075 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.075 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_lock_pp' 2024-06-18T13:04:54.075 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.076 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.076 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.076 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.076 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.076 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.076 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.076 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.076 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.076 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.076 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.076 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.077 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.077 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.077 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.077 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.077 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.077 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.077 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.077 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.077 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.077 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.077 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.077 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.078 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.078 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.078 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.078 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.078 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.078 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.078 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.078 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.078 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.078 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.078 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.079 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.079 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.079 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.079 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.079 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.079 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.079 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.079 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.079 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.079 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.079 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.079 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.079 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.080 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.080 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.080 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.080 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.080 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.080 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.080 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.080 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.080 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.080 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.080 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.081 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.081 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_aio_pp: /' 2024-06-18T13:04:54.081 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.081 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.081 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.081 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.081 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.081 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.081 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.081 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.081 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_lock_pp 2024-06-18T13:04:54.081 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.082 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.082 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.082 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.082 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.082 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.082 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.082 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.082 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.082 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.082 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.082 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/73921/exe 2024-06-18T13:04:54.082 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.083 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.083 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.083 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.083 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.083 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.083 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.083 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.083 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.083 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.083 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.083 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.083 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.083 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.084 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.084 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.084 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.084 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.084 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.084 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.084 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.084 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.084 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.084 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.084 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.084 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.084 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.085 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-06-18T13:04:54.085 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.085 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.085 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2024-06-18T13:04:54.085 INFO:tasks.workunit.client.0.smithi167.stdout:test api_lock_pp on pid 74008 2024-06-18T13:04:54.088 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.089 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.089 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.089 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.089 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.089 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.089 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.089 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.089 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.089 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.089 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_lock_pp 2024-06-18T13:04:54.089 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74008 2024-06-18T13:04:54.089 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_lock_pp on pid 74008' 2024-06-18T13:04:54.090 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74008 2024-06-18T13:04:54.090 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.090 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.090 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.090 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.090 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.090 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.090 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.090 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.090 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.090 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.090 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.091 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.091 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.091 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.091 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.091 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.091 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.091 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_io: /' 2024-06-18T13:04:54.091 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.091 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.091 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.091 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.091 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.091 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.092 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.092 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.092 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_io.log 2024-06-18T13:04:54.092 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.092 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.092 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.092 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.092 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.092 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.092 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.092 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.092 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.094 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-06-18T13:04:54.094 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.094 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.094 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.094 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.094 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.094 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.094 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.094 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.094 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.094 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.094 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.095 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.095 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.095 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.095 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.095 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.095 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.095 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.095 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.095 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.095 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2024-06-18T13:04:54.095 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.095 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.095 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/73932/exe 2024-06-18T13:04:54.095 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.096 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.096 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.096 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.096 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.096 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.096 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.096 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.096 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.096 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.096 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.096 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.096 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_misc 2024-06-18T13:04:54.096 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.096 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_misc' 2024-06-18T13:04:54.097 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.097 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.097 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.097 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.097 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.097 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.097 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.097 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.097 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.097 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.097 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.097 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.097 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.098 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.098 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.098 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.098 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.098 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.098 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.098 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.098 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.098 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.098 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.098 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.098 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.099 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.099 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.099 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.099 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.099 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.099 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.099 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.099 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.099 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.099 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.099 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.099 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.099 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.099 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.100 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.100 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.100 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.100 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.100 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.100 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.100 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.100 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.100 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-06-18T13:04:54.100 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.100 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.100 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.100 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.100 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.101 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.101 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.101 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.101 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.101 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.101 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.101 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.101 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.101 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.101 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.101 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.101 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.101 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.102 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.102 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.102 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.102 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.102 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.102 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.102 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.102 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.102 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.102 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.102 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.102 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.103 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.103 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2024-06-18T13:04:54.103 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_io_pp: /' 2024-06-18T13:04:54.103 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_misc 2024-06-18T13:04:54.103 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/73948/exe 2024-06-18T13:04:54.103 INFO:tasks.workunit.client.0.smithi167.stdout:test api_misc on pid 74036 2024-06-18T13:04:54.106 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_misc 2024-06-18T13:04:54.106 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74036 2024-06-18T13:04:54.106 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_misc on pid 74036' 2024-06-18T13:04:54.107 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74036 2024-06-18T13:04:54.107 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.107 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.107 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.107 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.107 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.107 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.107 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.107 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.107 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.107 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.107 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.108 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.108 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.108 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.108 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.108 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.108 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.108 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.108 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.108 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.108 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.108 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.108 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.108 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.108 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.109 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.109 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-06-18T13:04:54.109 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.109 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.109 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.109 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.109 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.109 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.109 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.109 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.109 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.109 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.109 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.110 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.110 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.110 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.110 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.110 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.110 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.110 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.110 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.110 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.110 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.110 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.110 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.110 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_misc_pp 2024-06-18T13:04:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_misc_pp' 2024-06-18T13:04:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_asio.log 2024-06-18T13:04:54.112 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.112 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.112 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.112 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.112 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.112 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.112 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.112 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.112 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2024-06-18T13:04:54.112 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.112 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.112 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.113 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.113 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.113 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.113 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.113 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.113 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.113 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.113 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.113 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.113 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.113 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.113 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.113 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.113 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.114 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.114 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.114 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.114 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.114 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.114 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.114 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.114 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.114 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.114 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.114 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.114 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.114 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.115 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.115 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.115 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.115 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.115 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.115 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.115 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.115 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.115 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.115 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.115 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.115 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.115 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.115 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.116 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.116 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.116 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.116 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.116 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.116 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.116 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.116 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.116 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.116 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.116 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.116 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.116 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.116 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.117 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.117 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.117 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.117 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.117 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.117 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.117 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.117 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.117 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.117 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.117 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.117 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.117 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_misc_pp 2024-06-18T13:04:54.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_asio: /' 2024-06-18T13:04:54.118 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.118 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.118 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.118 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.118 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.118 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.118 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.118 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.118 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.118 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.118 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.118 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.119 INFO:tasks.workunit.client.0.smithi167.stdout:test api_misc_pp on pid 74058 2024-06-18T13:04:54.122 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.122 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_misc_pp 2024-06-18T13:04:54.122 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.122 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.122 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.122 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.122 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74058 2024-06-18T13:04:54.122 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_misc_pp on pid 74058' 2024-06-18T13:04:54.122 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74058 2024-06-18T13:04:54.122 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.123 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_tier_pp 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_tier_pp' 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/73985/exe 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.124 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.125 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.125 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.125 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.125 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.125 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.125 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.125 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.125 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.125 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.125 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.125 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.125 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.125 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.125 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.126 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.126 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.126 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.126 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.126 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.126 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.126 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.126 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.126 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.126 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.126 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.126 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.126 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.126 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.127 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.127 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.127 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.127 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.127 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.127 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.127 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/73965/exe 2024-06-18T13:04:54.127 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2024-06-18T13:04:54.127 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.127 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.127 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.127 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.127 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.127 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.128 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.128 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.128 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.128 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_lock.log 2024-06-18T13:04:54.128 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.128 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.128 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.128 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.128 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.128 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.128 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.128 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.128 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.128 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.129 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.129 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.129 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.129 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.129 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.129 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.129 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_list: /' 2024-06-18T13:04:54.129 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2024-06-18T13:04:54.129 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.129 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.129 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.129 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.129 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.129 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.130 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.130 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.130 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.130 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.130 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.130 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.130 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.130 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.130 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.130 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.130 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.130 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.130 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.130 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.131 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.131 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.131 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.131 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.131 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.131 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.131 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.131 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.131 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.131 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_lock: /' 2024-06-18T13:04:54.131 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_list.log 2024-06-18T13:04:54.131 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_tier_pp 2024-06-18T13:04:54.131 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_tier_pp 2024-06-18T13:04:54.131 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74090 2024-06-18T13:04:54.132 INFO:tasks.workunit.client.0.smithi167.stdout:test api_tier_pp on pid 74090 2024-06-18T13:04:54.135 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_tier_pp on pid 74090' 2024-06-18T13:04:54.135 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74090 2024-06-18T13:04:54.136 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.136 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.136 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.136 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.136 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.136 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.136 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.136 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.136 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.136 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.136 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.136 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.136 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.137 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.137 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.137 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.137 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.137 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.137 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.137 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.137 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.137 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.137 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.137 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.137 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.137 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.137 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.138 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.138 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.138 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.138 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.138 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.138 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.138 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.138 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.138 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.138 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.138 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.138 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.138 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.139 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.140 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-06-18T13:04:54.140 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.140 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.140 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.140 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.140 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.140 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.140 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.140 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.140 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.140 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.140 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.140 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.140 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.141 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.141 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.141 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.141 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.141 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.141 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.141 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.141 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.141 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.141 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.141 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.141 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.141 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.142 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.142 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.142 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.142 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.142 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.142 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.142 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.142 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.142 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.142 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.142 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.142 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.142 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.142 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.143 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.143 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.143 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.143 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_pool 2024-06-18T13:04:54.143 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.143 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_pool' 2024-06-18T13:04:54.143 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.143 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.143 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.143 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.143 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.143 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.143 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74008/exe 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74036/exe 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.144 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.145 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.145 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.145 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.145 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.145 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.145 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.145 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.145 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.145 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.145 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.145 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.145 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.145 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.146 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2024-06-18T13:04:54.147 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_lock_pp: /' 2024-06-18T13:04:54.148 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_pool 2024-06-18T13:04:54.148 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-06-18T13:04:54.148 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_misc.log 2024-06-18T13:04:54.148 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2024-06-18T13:04:54.148 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74058/exe 2024-06-18T13:04:54.148 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_misc: /' 2024-06-18T13:04:54.148 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.148 INFO:tasks.workunit.client.0.smithi167.stdout:test api_pool on pid 74156 2024-06-18T13:04:54.152 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.153 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.153 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.153 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.153 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.153 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.153 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.153 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.153 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.153 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.153 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.153 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_pool 2024-06-18T13:04:54.153 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74156 2024-06-18T13:04:54.153 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_pool on pid 74156' 2024-06-18T13:04:54.153 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74156 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.154 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_snapshots 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_misc_pp: /' 2024-06-18T13:04:54.155 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_snapshots' 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.156 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.157 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_snapshots 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.158 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.159 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.159 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.159 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.159 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.159 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.159 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.159 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.159 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.159 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.159 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.159 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.159 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.159 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.159 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.160 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.160 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.160 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.160 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.160 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.160 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.160 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.160 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.160 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.160 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.160 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.160 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.160 INFO:tasks.workunit.client.0.smithi167.stdout:test api_snapshots on pid 74224 2024-06-18T13:04:54.163 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_snapshots 2024-06-18T13:04:54.163 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74224 2024-06-18T13:04:54.163 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_snapshots on pid 74224' 2024-06-18T13:04:54.163 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74224 2024-06-18T13:04:54.163 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.163 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.163 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-06-18T13:04:54.163 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.163 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.163 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.163 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.163 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.163 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.164 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_snapshots_pp 2024-06-18T13:04:54.164 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_snapshots_pp' 2024-06-18T13:04:54.164 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_snapshots_pp 2024-06-18T13:04:54.164 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.164 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.164 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.164 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.164 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.164 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.164 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.164 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.164 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.164 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.164 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.165 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.165 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.165 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.165 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.165 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.165 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.165 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.165 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.165 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.165 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.165 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.165 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.165 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.165 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.166 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.166 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.166 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.166 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.166 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.166 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.166 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.166 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.166 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.166 INFO:tasks.workunit.client.0.smithi167.stdout:test api_snapshots_pp on pid 74235 2024-06-18T13:04:54.169 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_snapshots_pp 2024-06-18T13:04:54.169 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74235 2024-06-18T13:04:54.169 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_snapshots_pp on pid 74235' 2024-06-18T13:04:54.169 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74235 2024-06-18T13:04:54.169 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.169 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.169 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.169 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-06-18T13:04:54.169 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.169 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.169 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.170 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.170 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.170 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.170 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.170 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.170 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.170 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.170 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.170 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.170 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.170 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.170 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.170 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.170 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.171 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.172 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_stat 2024-06-18T13:04:54.173 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.173 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_stat' 2024-06-18T13:04:54.173 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.173 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.173 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.173 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.173 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.173 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.173 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.173 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.173 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.173 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.173 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.173 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.174 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74090/exe 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_stat 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.175 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.176 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.176 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.176 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.176 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.176 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.176 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.176 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.176 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.176 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.176 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.176 INFO:tasks.workunit.client.0.smithi167.stdout:test api_stat on pid 74264 2024-06-18T13:04:54.180 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.180 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.180 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.180 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.180 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.180 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.180 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.180 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_stat 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74264 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_stat on pid 74264' 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74264 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.181 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.182 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.182 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.182 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.182 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.182 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.182 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.182 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.182 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.182 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.182 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.182 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.182 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.182 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.182 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.183 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_stat_pp 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_stat_pp' 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_tier_pp: /' 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.185 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74156/exe 2024-06-18T13:04:54.186 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.186 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.186 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.186 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.186 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.186 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.186 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.186 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.186 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_stat_pp 2024-06-18T13:04:54.186 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.186 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.186 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.186 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.186 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.187 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.188 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.188 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.188 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.188 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2024-06-18T13:04:54.188 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.188 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.188 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.188 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.188 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.188 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.188 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.188 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.188 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.188 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.189 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.189 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.189 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.189 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_stat_pp 2024-06-18T13:04:54.189 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74303 2024-06-18T13:04:54.189 INFO:tasks.workunit.client.0.smithi167.stdout:test api_stat_pp on pid 74303 2024-06-18T13:04:54.190 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_stat_pp on pid 74303' 2024-06-18T13:04:54.190 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74303 2024-06-18T13:04:54.190 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.190 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.190 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_watch_notify 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_watch_notify' 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_pool.log 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.191 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.192 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.192 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.192 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.192 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.192 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.192 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.192 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.192 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.192 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.192 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.192 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.192 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.192 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.192 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_pool: /' 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.193 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.194 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.194 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.194 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.194 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.194 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.194 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.194 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.194 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.194 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_watch_notify 2024-06-18T13:04:54.194 INFO:tasks.workunit.client.0.smithi167.stdout:test api_watch_notify on pid 74317 2024-06-18T13:04:54.198 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.198 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_watch_notify 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.199 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74317 2024-06-18T13:04:54.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_watch_notify on pid 74317' 2024-06-18T13:04:54.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74317 2024-06-18T13:04:54.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.200 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_watch_notify_pp 2024-06-18T13:04:54.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-06-18T13:04:54.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_watch_notify_pp' 2024-06-18T13:04:54.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-06-18T13:04:54.200 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_watch_notify_pp 2024-06-18T13:04:54.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.200 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.201 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.202 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.202 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.202 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.202 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.202 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.202 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.202 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.202 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.202 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.202 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.202 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.202 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.202 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.203 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.203 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.203 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.203 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.203 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.203 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.203 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.203 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.203 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.203 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.203 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.203 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74224/exe 2024-06-18T13:04:54.203 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.203 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.204 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.204 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.204 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.204 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.204 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.204 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.204 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.204 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.204 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.204 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.204 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.204 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.204 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.205 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.205 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.205 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.205 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.205 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.205 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.205 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.205 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.205 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.205 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.205 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.205 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.205 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.205 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.206 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.206 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.206 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.206 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.206 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.206 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.206 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2024-06-18T13:04:54.206 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.206 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.206 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.206 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.206 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.206 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_snapshots: /' 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.207 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.208 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.209 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.210 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.210 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.210 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.210 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.210 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.210 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.210 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.210 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.210 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.210 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.210 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.210 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.210 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.210 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.211 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.211 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_watch_notify_pp 2024-06-18T13:04:54.211 INFO:tasks.workunit.client.0.smithi167.stdout:test api_watch_notify_pp on pid 74353 2024-06-18T13:04:54.213 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74353 2024-06-18T13:04:54.213 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_watch_notify_pp on pid 74353' 2024-06-18T13:04:54.213 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74353 2024-06-18T13:04:54.213 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.213 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_cmd 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.214 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.215 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.215 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_cmd' 2024-06-18T13:04:54.215 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.215 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.215 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.215 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.215 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.215 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.215 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.215 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_cmd 2024-06-18T13:04:54.215 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.215 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.215 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.215 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.216 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74235/exe 2024-06-18T13:04:54.218 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.218 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.218 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.218 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.218 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.218 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.218 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.218 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.218 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.218 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.218 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.218 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.218 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.218 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.219 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.220 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.220 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.220 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.220 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.220 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2024-06-18T13:04:54.220 INFO:tasks.workunit.client.0.smithi167.stdout:test api_cmd on pid 74382 2024-06-18T13:04:54.225 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_cmd 2024-06-18T13:04:54.225 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74382 2024-06-18T13:04:54.225 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_cmd on pid 74382' 2024-06-18T13:04:54.225 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74382 2024-06-18T13:04:54.225 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.225 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.225 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.226 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_cmd_pp 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_cmd_pp' 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.227 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.228 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.228 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.228 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.228 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-06-18T13:04:54.228 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-06-18T13:04:54.228 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.228 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.228 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.228 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.228 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.228 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.228 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.228 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.228 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.229 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.229 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.229 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.229 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.229 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.229 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.229 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.229 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.229 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.229 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.229 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.229 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.229 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.229 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_cmd_pp 2024-06-18T13:04:54.230 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.230 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.230 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.230 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.230 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.230 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.230 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.230 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.230 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.230 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.230 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.230 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.230 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74303/exe 2024-06-18T13:04:54.231 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.232 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.232 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.232 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.232 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74264/exe 2024-06-18T13:04:54.232 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.232 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.232 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.232 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.232 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.232 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.232 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.232 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.232 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.233 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.234 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.235 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.235 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.235 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.235 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.235 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.235 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.235 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.235 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.235 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.235 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.235 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.235 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.235 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.235 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_stat: /' 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.236 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.237 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.237 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.237 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.237 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_cmd_pp 2024-06-18T13:04:54.237 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74419 2024-06-18T13:04:54.237 INFO:tasks.workunit.client.0.smithi167.stdout:test api_cmd_pp on pid 74419 2024-06-18T13:04:54.240 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_cmd_pp on pid 74419' 2024-06-18T13:04:54.240 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74419 2024-06-18T13:04:54.241 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.241 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.241 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.241 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.241 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.241 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.241 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.241 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.241 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.241 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.241 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-06-18T13:04:54.241 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.241 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.242 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.243 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.243 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.243 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.243 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.243 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.243 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.243 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.243 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.243 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.243 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.243 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.243 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.243 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.243 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.244 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.244 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.244 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.244 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.244 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.244 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.244 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.244 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.244 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_service 2024-06-18T13:04:54.244 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_service' 2024-06-18T13:04:54.244 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.244 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.244 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.245 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.246 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.246 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_stat_pp: /' 2024-06-18T13:04:54.246 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.246 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.246 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.246 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.246 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.246 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.246 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.246 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.246 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.246 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.246 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-06-18T13:04:54.246 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_stat.log 2024-06-18T13:04:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.248 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.248 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.248 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.248 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74317/exe 2024-06-18T13:04:54.248 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.248 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.248 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.248 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.248 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.248 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.248 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.248 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.248 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.248 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.249 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_service 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.250 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.251 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.251 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.251 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.251 INFO:tasks.workunit.client.0.smithi167.stdout:test api_service on pid 74465 2024-06-18T13:04:54.255 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_service 2024-06-18T13:04:54.255 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74465 2024-06-18T13:04:54.255 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_service on pid 74465' 2024-06-18T13:04:54.255 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.256 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74465 2024-06-18T13:04:54.256 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.256 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.256 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.256 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.256 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.256 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.256 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.256 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.256 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.256 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.256 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.256 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.257 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.257 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.257 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.257 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.257 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.257 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.257 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.257 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.257 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.257 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.257 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-06-18T13:04:54.257 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.257 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.258 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.259 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.259 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.259 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.259 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.259 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.259 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.259 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.259 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.259 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.259 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.259 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.259 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.259 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.259 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.260 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_service_pp 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_service_pp' 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.261 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_watch_notify: /' 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.262 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.263 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.263 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.263 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.263 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74382/exe 2024-06-18T13:04:54.263 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.263 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74353/exe 2024-06-18T13:04:54.263 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_service_pp 2024-06-18T13:04:54.263 INFO:tasks.workunit.client.0.smithi167.stdout:test api_service_pp on pid 74502 2024-06-18T13:04:54.265 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.265 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.265 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_service_pp 2024-06-18T13:04:54.265 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.265 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.266 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.266 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.266 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.266 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74502 2024-06-18T13:04:54.266 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_service_pp on pid 74502' 2024-06-18T13:04:54.266 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74502 2024-06-18T13:04:54.266 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.266 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.266 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.266 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.266 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.266 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.267 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_cmd.log 2024-06-18T13:04:54.267 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2024-06-18T13:04:54.267 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_cmd: /' 2024-06-18T13:04:54.267 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.267 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.267 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.267 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.267 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.267 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.267 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.267 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-06-18T13:04:54.267 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.267 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.268 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.268 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_c_write_operations 2024-06-18T13:04:54.268 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_c_write_operations' 2024-06-18T13:04:54.268 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-06-18T13:04:54.268 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.268 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.268 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.268 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.268 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.268 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.268 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.268 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.268 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.268 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.269 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.269 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.269 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.269 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.269 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.269 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.269 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.269 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.269 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.269 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.269 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-06-18T13:04:54.269 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2024-06-18T13:04:54.269 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.269 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.270 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.270 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.270 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.270 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.270 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.270 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.270 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.270 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.270 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.270 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.270 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.270 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.270 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.270 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.271 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.272 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.272 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.272 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.272 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.272 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.272 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.272 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.272 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.272 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.272 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.272 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.272 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.272 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.272 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.273 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.273 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.273 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.273 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.273 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.273 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.273 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.273 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_c_write_operations 2024-06-18T13:04:54.273 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_c_write_operations 2024-06-18T13:04:54.273 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74547 2024-06-18T13:04:54.273 INFO:tasks.workunit.client.0.smithi167.stdout:test api_c_write_operations on pid 74547 2024-06-18T13:04:54.277 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_c_write_operations on pid 74547' 2024-06-18T13:04:54.277 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74547 2024-06-18T13:04:54.277 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.277 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.277 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.277 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.277 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.277 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.277 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.278 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.278 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.278 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.278 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.278 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.278 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.278 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.278 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.278 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.278 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.278 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.278 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.278 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.279 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_c_read_operations 2024-06-18T13:04:54.280 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_c_read_operations' 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.281 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.282 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.282 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.282 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.282 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.282 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.282 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.282 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.282 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.282 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.282 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.282 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.282 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.282 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.282 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_c_read_operations 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.283 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74419/exe 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.284 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.285 INFO:tasks.workunit.client.0.smithi167.stdout:test api_c_read_operations on pid 74581 2024-06-18T13:04:54.287 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.287 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.287 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.287 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.287 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.287 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.287 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.287 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.287 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.287 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_c_read_operations 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74581 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_c_read_operations on pid 74581' 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74581 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s api_cls_remote_reads 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-06-18T13:04:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.290 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.290 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.290 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' api_cls_remote_reads' 2024-06-18T13:04:54.290 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.290 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.290 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.290 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo api_cls_remote_reads 2024-06-18T13:04:54.290 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-06-18T13:04:54.290 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.290 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.290 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.290 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.290 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.290 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.291 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2024-06-18T13:04:54.292 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.292 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.292 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.292 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74465/exe 2024-06-18T13:04:54.292 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.292 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.292 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.292 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.292 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.292 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.292 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.292 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.292 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_service: /' 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.293 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.294 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.294 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.294 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.294 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.294 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.294 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.294 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.294 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.294 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.294 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.294 INFO:tasks.workunit.client.0.smithi167.stdout:test api_cls_remote_reads on pid 74614 2024-06-18T13:04:54.296 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=api_cls_remote_reads 2024-06-18T13:04:54.296 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74614 2024-06-18T13:04:54.296 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test api_cls_remote_reads on pid 74614' 2024-06-18T13:04:54.296 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74614 2024-06-18T13:04:54.297 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.297 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.297 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s list_parallel 2024-06-18T13:04:54.297 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.297 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' list_parallel' 2024-06-18T13:04:54.297 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-06-18T13:04:54.297 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.297 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.297 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.297 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.297 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.297 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.297 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.298 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.298 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.298 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.298 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.298 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.298 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.298 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.298 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.298 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.298 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.298 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.298 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.298 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.298 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo list_parallel 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.299 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.300 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.300 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.300 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.300 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.300 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.300 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.300 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.300 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.300 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=list_parallel 2024-06-18T13:04:54.300 INFO:tasks.workunit.client.0.smithi167.stdout:test list_parallel on pid 74625 2024-06-18T13:04:54.303 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.303 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.303 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.303 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.303 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74625 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_service.log 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test list_parallel on pid 74625' 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74625 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s open_pools_parallel 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' open_pools_parallel' 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.305 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.306 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.306 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.306 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.306 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.306 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.306 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.306 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.306 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.306 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.306 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.306 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-06-18T13:04:54.306 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74502/exe 2024-06-18T13:04:54.306 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.307 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.307 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.307 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.307 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.307 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.307 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.307 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.307 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2024-06-18T13:04:54.307 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.307 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.307 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.307 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.307 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.307 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.308 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.308 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.308 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.308 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.308 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.308 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.308 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.308 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.308 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.308 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.308 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.308 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.308 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.308 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.309 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.309 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.309 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo open_pools_parallel 2024-06-18T13:04:54.309 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.309 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.309 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.309 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.309 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.309 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.309 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.309 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.309 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.309 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.310 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.311 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.311 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.311 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.311 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.311 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.311 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.311 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.311 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.311 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.311 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.311 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.311 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.311 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.311 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_service_pp: /' 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=open_pools_parallel 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.312 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.313 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.313 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.313 INFO:tasks.workunit.client.0.smithi167.stdout:test open_pools_parallel on pid 74656 2024-06-18T13:04:54.314 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74656 2024-06-18T13:04:54.314 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test open_pools_parallel on pid 74656' 2024-06-18T13:04:54.314 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74656 2024-06-18T13:04:54.314 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.314 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-18T13:04:54.314 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s delete_pools_parallel 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' delete_pools_parallel' 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.315 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.316 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.317 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.318 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.318 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.318 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.318 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.318 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo delete_pools_parallel 2024-06-18T13:04:54.318 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-06-18T13:04:54.318 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-06-18T13:04:54.318 INFO:tasks.workunit.client.0.smithi167.stdout:test delete_pools_parallel on pid 74678 2024-06-18T13:04:54.323 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=delete_pools_parallel 2024-06-18T13:04:54.323 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74678 2024-06-18T13:04:54.323 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test delete_pools_parallel on pid 74678' 2024-06-18T13:04:54.323 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74678 2024-06-18T13:04:54.323 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-18T13:04:54.324 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.324 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s cls 2024-06-18T13:04:54.324 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.324 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.324 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.324 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.324 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.324 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.324 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.324 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.324 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.324 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.324 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' cls' 2024-06-18T13:04:54.324 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.325 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74547/exe 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.326 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.327 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.327 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.327 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo cls 2024-06-18T13:04:54.327 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.327 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.327 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.327 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.327 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.327 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.327 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.327 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.327 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.327 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.327 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.328 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.328 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.328 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.328 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.328 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.328 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.328 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.328 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.328 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.328 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.328 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.328 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.328 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.328 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.329 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.330 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.330 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.330 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.330 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.330 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.330 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.330 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.330 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.330 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-06-18T13:04:54.330 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.330 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.330 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.330 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.330 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.331 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.331 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.331 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.331 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.331 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.331 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.331 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.331 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.331 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.331 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.331 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.331 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.331 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.332 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2024-06-18T13:04:54.332 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-06-18T13:04:54.332 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.332 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.332 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.332 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.332 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.332 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.332 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.332 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.332 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.332 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.332 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.332 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.333 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.333 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.333 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.333 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.333 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.333 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.333 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.333 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.333 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.333 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.333 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.333 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.333 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.333 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.334 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.334 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.334 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.334 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.334 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.334 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.334 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.334 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.334 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.334 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.334 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.334 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.334 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.334 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.335 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.335 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.335 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.335 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.335 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.335 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.335 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.335 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.335 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.335 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.335 INFO:tasks.workunit.client.0.smithi167.stdout:test cls on pid 74712 2024-06-18T13:04:54.340 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=cls 2024-06-18T13:04:54.340 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74712 2024-06-18T13:04:54.340 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test cls on pid 74712' 2024-06-18T13:04:54.340 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74712 2024-06-18T13:04:54.340 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-18T13:04:54.340 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.340 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.340 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.340 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.340 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.341 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.342 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.342 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.342 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.342 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.342 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.342 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.342 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.342 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.342 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.342 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.342 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.342 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.342 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.343 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.343 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.343 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74614/exe 2024-06-18T13:04:54.343 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-06-18T13:04:54.343 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.343 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.343 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.343 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.343 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.343 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.343 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.343 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.343 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.343 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.344 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.344 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.344 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.344 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.344 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.344 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.344 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.344 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.344 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.344 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.344 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.344 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74625/exe 2024-06-18T13:04:54.344 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.344 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.345 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.345 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.345 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.345 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.345 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.345 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.345 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.345 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.345 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.345 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.345 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.345 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.345 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.345 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.346 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.346 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.346 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.346 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.346 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.346 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.346 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.346 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.346 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.346 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.346 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.346 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.346 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.346 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ list_parallel: /' 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.347 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.348 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_list_parallel.log 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.349 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.350 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.350 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.350 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.350 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.350 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s cmd 2024-06-18T13:04:54.350 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.350 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.350 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.350 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.350 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.350 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.350 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.350 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.351 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.351 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.351 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' cmd' 2024-06-18T13:04:54.351 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.351 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.351 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.351 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.351 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.351 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.351 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2024-06-18T13:04:54.351 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.351 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.352 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.352 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.352 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.352 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.352 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.352 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.352 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.352 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.352 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.352 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.352 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.352 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.352 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.353 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.354 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.354 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.354 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.354 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.354 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.354 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.354 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.354 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.354 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.354 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.354 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.354 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.354 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.354 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74581/exe 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.355 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.356 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.356 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.356 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.356 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.356 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.356 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.356 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.356 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.356 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.356 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.356 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.356 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.356 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.357 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.357 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.357 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-06-18T13:04:54.357 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.357 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.357 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.357 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.357 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.357 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.357 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.357 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.357 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.357 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2024-06-18T13:04:54.357 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-06-18T13:04:54.358 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-06-18T13:04:54.358 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo cmd 2024-06-18T13:04:54.358 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.358 INFO:tasks.workunit.client.0.smithi167.stdout:test cmd on pid 74765 2024-06-18T13:04:54.363 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=cmd 2024-06-18T13:04:54.364 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74765 2024-06-18T13:04:54.364 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test cmd on pid 74765' 2024-06-18T13:04:54.364 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74765 2024-06-18T13:04:54.364 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-18T13:04:54.364 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.364 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.364 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.364 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.364 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.364 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.364 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.364 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.364 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.364 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.365 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.366 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.366 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.366 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.366 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.366 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.366 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.366 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.366 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.366 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s handler_error 2024-06-18T13:04:54.366 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' handler_error' 2024-06-18T13:04:54.366 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-06-18T13:04:54.366 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.366 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.366 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.367 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.367 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.367 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.367 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.367 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.367 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.367 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.367 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.367 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.367 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.367 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.367 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.367 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.368 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.369 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.370 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.370 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.370 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.370 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.370 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.370 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.370 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.370 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.370 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.370 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74712/exe 2024-06-18T13:04:54.370 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.370 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.370 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.372 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.372 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.372 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.372 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.372 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.372 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.372 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.372 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.372 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.372 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.372 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.372 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.372 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.372 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.373 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.373 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.373 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.373 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.373 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.373 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.373 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.373 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.373 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.373 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.373 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.373 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.373 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.373 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.374 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.374 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.374 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.374 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo handler_error 2024-06-18T13:04:54.374 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_neorados_cls 2024-06-18T13:04:54.374 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.374 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.374 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.374 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.374 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.374 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.374 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_neorados_cls.log 2024-06-18T13:04:54.375 INFO:tasks.workunit.client.0.smithi167.stdout:test handler_error on pid 74826 2024-06-18T13:04:54.378 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=handler_error 2024-06-18T13:04:54.378 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ cls: /' 2024-06-18T13:04:54.378 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74826 2024-06-18T13:04:54.378 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test handler_error on pid 74826' 2024-06-18T13:04:54.378 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74826 2024-06-18T13:04:54.378 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-18T13:04:54.378 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.378 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s io 2024-06-18T13:04:54.378 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' io' 2024-06-18T13:04:54.378 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-06-18T13:04:54.378 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.378 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74678/exe 2024-06-18T13:04:54.379 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.379 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.379 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.379 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.379 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.379 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.379 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.379 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.379 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.379 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.379 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.379 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.379 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.379 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.380 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.380 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.380 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.380 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.380 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.380 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.380 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.380 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.380 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.380 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.380 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.380 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.380 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.380 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.381 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.381 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.381 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.381 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.381 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.381 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.381 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.381 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.381 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.381 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.381 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.381 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.381 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.382 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.382 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.382 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.382 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.382 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.382 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.382 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.382 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.382 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.382 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.382 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.382 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.382 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo io 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.383 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.384 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.384 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.384 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.384 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.384 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.384 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.384 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.384 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.384 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.384 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.384 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74656/exe 2024-06-18T13:04:54.384 INFO:tasks.workunit.client.0.smithi167.stdout:test io on pid 74881 2024-06-18T13:04:54.388 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=io 2024-06-18T13:04:54.389 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74881 2024-06-18T13:04:54.389 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test io on pid 74881' 2024-06-18T13:04:54.389 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74881 2024-06-18T13:04:54.389 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-18T13:04:54.389 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.389 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.389 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.389 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.389 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.389 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.389 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.389 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.389 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2024-06-18T13:04:54.390 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.390 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.390 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.390 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.390 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.390 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.390 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.390 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.390 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.390 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.390 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.390 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.390 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.390 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.391 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.391 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.391 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.391 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.391 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.391 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.391 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.391 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.391 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.391 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.391 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-06-18T13:04:54.391 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s ec_io 2024-06-18T13:04:54.391 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' ec_io' 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.392 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.393 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.393 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.393 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.393 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.393 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.393 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.393 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo ec_io 2024-06-18T13:04:54.393 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.393 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.393 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.393 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.393 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.393 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.393 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.394 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.394 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.394 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.394 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.394 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.394 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.394 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.394 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.394 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.394 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.394 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.394 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.394 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.395 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.395 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.395 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.395 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.395 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.395 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.395 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.395 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.395 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.395 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.395 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.395 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.395 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.395 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.396 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.396 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.396 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.396 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.396 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.396 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.396 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.396 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.396 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.396 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.396 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.396 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=ec_io 2024-06-18T13:04:54.396 INFO:tasks.workunit.client.0.smithi167.stdout:test ec_io on pid 74954 2024-06-18T13:04:54.400 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74954 2024-06-18T13:04:54.400 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test ec_io on pid 74954' 2024-06-18T13:04:54.400 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=74954 2024-06-18T13:04:54.401 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-18T13:04:54.401 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.401 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s list 2024-06-18T13:04:54.401 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' list' 2024-06-18T13:04:54.401 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.401 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-06-18T13:04:54.401 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.401 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.401 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.401 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.401 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.401 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.401 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.401 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.402 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.402 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.402 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.402 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.402 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.402 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.402 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.402 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.402 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.402 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.402 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.402 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.402 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo list 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.403 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.404 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.404 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.404 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.404 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.404 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.404 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.404 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.404 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.404 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.404 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.404 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.404 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.404 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.404 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.405 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.406 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.406 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.406 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.406 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.406 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.406 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.406 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.406 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.406 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.406 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.406 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.406 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.406 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.406 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.407 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.407 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.407 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.407 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.407 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.407 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.407 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.407 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.407 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.407 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.407 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.407 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.407 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.407 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.408 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.408 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.408 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.408 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.408 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.408 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.408 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.408 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.408 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=list 2024-06-18T13:04:54.408 INFO:tasks.workunit.client.0.smithi167.stdout:test list on pid 75004 2024-06-18T13:04:54.412 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75004 2024-06-18T13:04:54.413 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test list on pid 75004' 2024-06-18T13:04:54.413 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=75004 2024-06-18T13:04:54.413 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-18T13:04:54.413 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.413 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-06-18T13:04:54.413 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.413 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.413 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.413 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.413 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.413 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74765/exe 2024-06-18T13:04:54.413 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.414 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.414 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.414 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.414 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.414 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.414 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.414 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.414 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.414 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.414 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.414 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.414 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.414 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.415 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.415 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.415 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.415 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.415 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.415 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.415 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.415 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.415 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.415 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.415 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.415 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.415 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.415 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.416 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.416 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.416 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.416 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.416 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.416 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.416 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.416 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.416 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.416 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.416 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.416 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.416 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.417 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.417 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.417 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.417 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.417 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.417 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.417 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.417 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.417 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.417 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.417 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.417 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.417 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.417 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.418 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.419 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.419 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.419 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.419 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.419 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.419 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.419 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.419 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.419 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.419 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.419 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74826/exe 2024-06-18T13:04:54.419 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.419 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.419 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_neorados_cmd.log 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ cmd: /' 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.420 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_neorados_cmd 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.421 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.422 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.422 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.422 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.422 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.422 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.422 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.422 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.422 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.422 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.422 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.422 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ handler_error: /' 2024-06-18T13:04:54.422 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_neorados_handler_error 2024-06-18T13:04:54.422 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.422 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_neorados_handler_error.log 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.423 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74881/exe 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s ec_list 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' ec_list' 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.424 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.425 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.425 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_neorados_io.log 2024-06-18T13:04:54.425 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo ec_list 2024-06-18T13:04:54.425 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.425 INFO:tasks.workunit.client.0.smithi167.stdout:test ec_list on pid 75034 2024-06-18T13:04:54.428 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=ec_list 2024-06-18T13:04:54.428 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75034 2024-06-18T13:04:54.428 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test ec_list on pid 75034' 2024-06-18T13:04:54.428 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=75034 2024-06-18T13:04:54.428 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-18T13:04:54.428 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.428 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-06-18T13:04:54.428 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.428 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.428 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.428 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.429 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.429 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.429 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.429 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.429 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.429 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.429 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.429 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.429 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.429 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.429 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.429 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.429 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.429 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.430 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.430 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.430 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.430 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.430 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.430 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.430 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.430 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.430 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.430 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.430 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.430 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.430 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.431 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.431 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.431 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.431 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.431 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.431 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.431 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.431 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.431 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.431 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_neorados_io 2024-06-18T13:04:54.431 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.431 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.431 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.431 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.432 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.432 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.432 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.432 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.432 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.432 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.432 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.432 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.432 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.432 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ io: /' 2024-06-18T13:04:54.432 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.432 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.432 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.432 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.433 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.433 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.433 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.433 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.433 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.433 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.433 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.433 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.433 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.433 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.433 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.433 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.433 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.434 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.434 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.434 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.434 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.434 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.434 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.434 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.434 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.434 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.434 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.434 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.434 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.434 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.434 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/74954/exe 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.435 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.436 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.436 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.436 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.436 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_neorados_ec_io 2024-06-18T13:04:54.436 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ ec_io: /' 2024-06-18T13:04:54.436 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_neorados_ec_io.log 2024-06-18T13:04:54.436 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s misc 2024-06-18T13:04:54.436 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' misc' 2024-06-18T13:04:54.436 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.436 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.436 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.436 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.436 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.436 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.437 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.437 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.437 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.437 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo misc 2024-06-18T13:04:54.437 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.437 INFO:tasks.workunit.client.0.smithi167.stdout:test misc on pid 75086 2024-06-18T13:04:54.442 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=misc 2024-06-18T13:04:54.443 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75086 2024-06-18T13:04:54.443 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test misc on pid 75086' 2024-06-18T13:04:54.443 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=75086 2024-06-18T13:04:54.443 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-18T13:04:54.443 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.443 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-06-18T13:04:54.443 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.443 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.443 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.443 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.443 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.443 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.443 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.444 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.444 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.444 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.444 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.444 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.444 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.444 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.444 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.444 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.444 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.444 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.444 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.444 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.444 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.445 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.445 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.445 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.445 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.445 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.445 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.445 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.445 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.445 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.445 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.445 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.445 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.445 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.447 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.449 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.449 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.449 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.449 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.449 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.449 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.449 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.449 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.449 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.449 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.449 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.449 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.449 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.449 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.450 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.450 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.450 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.450 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.450 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.450 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.450 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.450 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.450 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.450 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.450 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.450 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.450 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.451 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.451 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.451 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.451 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.451 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.451 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/75004/exe 2024-06-18T13:04:54.451 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.451 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s pool 2024-06-18T13:04:54.451 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.451 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.451 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.451 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.451 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.452 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.452 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.452 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.452 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.452 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.452 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.452 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.452 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.452 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.452 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.452 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.452 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.452 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.453 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' pool' 2024-06-18T13:04:54.453 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.453 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.453 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.453 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.453 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.453 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.453 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.453 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.453 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.453 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.453 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ list: /' 2024-06-18T13:04:54.453 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_neorados_list.log 2024-06-18T13:04:54.453 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.454 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.454 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.454 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.454 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.454 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.454 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.454 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.454 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.454 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.454 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.454 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.454 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.455 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.455 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.455 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.455 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.455 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.455 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_neorados_list 2024-06-18T13:04:54.455 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.455 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.455 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.455 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.455 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.455 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.455 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.456 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.456 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.456 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.456 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.456 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.456 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo pool 2024-06-18T13:04:54.456 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.456 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.456 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.456 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.456 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.456 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.456 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.456 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.457 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.457 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.457 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.457 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.457 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.457 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.457 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.457 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.457 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.457 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.457 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.457 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.457 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.458 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.459 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.459 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.459 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.459 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.459 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.459 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.459 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.459 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.459 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.459 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.459 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.459 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.459 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.459 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.460 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.460 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=pool 2024-06-18T13:04:54.460 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.460 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.460 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.460 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.460 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.460 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.460 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.460 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.460 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.460 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75192 2024-06-18T13:04:54.460 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.460 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test pool on pid 75192' 2024-06-18T13:04:54.461 INFO:tasks.workunit.client.0.smithi167.stdout:test pool on pid 75192 2024-06-18T13:04:54.465 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=75192 2024-06-18T13:04:54.465 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-18T13:04:54.465 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/75034/exe 2024-06-18T13:04:54.465 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.465 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-06-18T13:04:54.465 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/75086/exe 2024-06-18T13:04:54.465 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.466 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.466 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.466 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.466 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.466 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.466 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.466 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.466 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s read_operations 2024-06-18T13:04:54.466 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.466 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.466 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.466 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.466 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.466 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.467 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.467 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' read_operations' 2024-06-18T13:04:54.467 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.467 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.467 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.467 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.467 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.467 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.467 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.467 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.467 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.467 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.467 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.467 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_neorados_ec_list 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo read_operations 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_neorados_ec_list.log 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ misc: /' 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.468 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ ec_list: /' 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_neorados_misc 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_neorados_misc.log 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.469 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.470 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.470 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.470 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.470 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.470 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.470 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.470 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.470 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.470 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.470 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.470 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=read_operations 2024-06-18T13:04:54.470 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.470 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.470 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.471 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.471 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.471 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.471 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.471 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75217 2024-06-18T13:04:54.471 INFO:tasks.workunit.client.0.smithi167.stdout:test read_operations on pid 75217 2024-06-18T13:04:54.473 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test read_operations on pid 75217' 2024-06-18T13:04:54.474 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=75217 2024-06-18T13:04:54.474 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-18T13:04:54.474 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.474 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-06-18T13:04:54.474 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s snapshots 2024-06-18T13:04:54.474 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' snapshots' 2024-06-18T13:04:54.474 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo snapshots 2024-06-18T13:04:54.474 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.474 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.474 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.474 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.474 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.475 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.476 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.477 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=snapshots 2024-06-18T13:04:54.477 INFO:tasks.workunit.client.0.smithi167.stdout:test snapshots on pid 75227 2024-06-18T13:04:54.480 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75227 2024-06-18T13:04:54.480 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test snapshots on pid 75227' 2024-06-18T13:04:54.480 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=75227 2024-06-18T13:04:54.480 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-18T13:04:54.480 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.480 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-06-18T13:04:54.480 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s watch_notify 2024-06-18T13:04:54.480 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' watch_notify' 2024-06-18T13:04:54.480 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo watch_notify 2024-06-18T13:04:54.480 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.480 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.481 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.481 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.481 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.481 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.481 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.481 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.481 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.481 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.481 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.481 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.481 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.481 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.481 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.481 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.482 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.482 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.482 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.482 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.482 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.482 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.482 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.482 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.482 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.482 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.482 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.482 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.482 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.482 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.483 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.484 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.485 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.485 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.485 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.485 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.485 INFO:tasks.workunit.client.0.smithi167.stdout:test watch_notify on pid 75239 2024-06-18T13:04:54.486 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=watch_notify 2024-06-18T13:04:54.486 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75239 2024-06-18T13:04:54.487 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test watch_notify on pid 75239' 2024-06-18T13:04:54.487 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=75239 2024-06-18T13:04:54.487 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-18T13:04:54.487 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.487 INFO:tasks.workunit.client.0.smithi167.stderr:++ printf %25s write_operations 2024-06-18T13:04:54.487 INFO:tasks.workunit.client.0.smithi167.stderr:+ r=' write_operations' 2024-06-18T13:04:54.487 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.487 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.487 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.487 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.487 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.487 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1}' 2024-06-18T13:04:54.487 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-06-18T13:04:54.487 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.488 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.489 INFO:tasks.workunit.client.0.smithi167.stderr:++ echo write_operations 2024-06-18T13:04:54.490 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.490 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.490 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.490 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.490 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.490 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.490 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.490 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.490 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.490 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.490 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.490 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.490 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.490 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.491 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.491 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.491 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.491 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.491 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.491 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.491 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.491 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/75192/exe 2024-06-18T13:04:54.491 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.491 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.491 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.491 INFO:tasks.workunit.client.0.smithi167.stdout:test write_operations on pid 75253 2024-06-18T13:04:54.500 INFO:tasks.workunit.client.0.smithi167.stderr:+ ff=write_operations 2024-06-18T13:04:54.500 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75253 2024-06-18T13:04:54.500 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'test write_operations on pid 75253' 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:+ pids[$f]=75253 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:+ ret=0 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 1 ']' 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74090 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74090 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.501 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.502 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.502 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.502 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.502 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.502 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.502 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.502 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.502 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.502 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.502 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.502 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.502 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.502 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.502 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.503 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.504 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.504 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.504 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.504 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.504 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.504 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.504 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.504 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.504 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.504 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.504 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.504 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.504 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.504 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_neorados_pool.log 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_neorados_pool 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ pool: /' 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -f /etc/bashrc ']' 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /etc/bashrc 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -z '' ']' 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:++ BASHRCSOURCED=Y 2024-06-18T13:04:54.505 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:++ shopt -q login_shell 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:+++ umask 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' 0022 -eq 0 ']' 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:++ SHELL=/bin/bash 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.506 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.507 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.508 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.509 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'grep=grep --color=auto' 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.510 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorls.sh 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' '!' -t 0 ']' 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:+++ return 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export S_COLORS=auto 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:+++ S_COLORS=auto 2024-06-18T13:04:54.511 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.512 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.513 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.514 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/75227/exe 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/75217/exe 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.515 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.516 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_neorados_snapshots 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_neorados_read_operations 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-18T13:04:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.518 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-18T13:04:54.518 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-18T13:04:54.518 INFO:tasks.workunit.client.0.smithi167.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-18T13:04:54.518 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_neorados_snapshots.log 2024-06-18T13:04:54.518 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ snapshots: /' 2024-06-18T13:04:54.518 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_neorados_read_operations.log 2024-06-18T13:04:54.518 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.518 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ read_operations: /' 2024-06-18T13:04:54.518 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.518 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.518 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.518 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.519 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-18T13:04:54.520 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/gawk.sh 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/lang.sh 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/75239/exe 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/locale 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.521 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_neorados_watch_notify 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-06-18T13:04:54.522 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ watch_notify: /' 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:++++ export LANG=en_US.UTF-8 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:++++ LANG=en_US.UTF-8 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:+++ LANG=en_US.UTF-8 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:+++ unset LANG_backup config 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n '' ']' 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' dumb = linux ']' 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/less.sh 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-18T13:04:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:++ for i in /etc/profile.d/*.sh 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' '' ']' 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:++ . /etc/profile.d/which2.sh 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:+++++ readlink /proc/75253/exe 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:++++ basename /usr/bin/bash 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_declare='declare -f' 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:+++ which_opt=-f 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export which_declare 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:+++ export -f which 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset i 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:++ unset -f pathmunge 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:+ export PATH 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:+ unset rc 2024-06-18T13:04:54.524 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph_test_neorados_write_operations 2024-06-18T13:04:54.525 INFO:tasks.workunit.client.0.smithi167.stderr:+ tee ceph_test_neorados_write_operations.log 2024-06-18T13:04:54.525 INFO:tasks.workunit.client.0.smithi167.stderr:+ sed 's/^/ write_operations: /' 2024-06-18T13:04:54.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/156734772' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-06-18T13:04:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:04:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/156734772' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-06-18T13:04:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:04:55.008 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/156734772' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-06-18T13:04:55.008 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:04:55.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:04:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:04:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:04:56.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:55 smithi179 ceph-mon[33312]: pgmap v58: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2835616045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi167-75207-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:04:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4052815225' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi167-73991-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:04:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:55 smithi179 ceph-mon[33312]: osdmap e74: 8 total, 8 up, 8 in 2024-06-18T13:04:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:04:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/309801237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi167-75055-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:04:56.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:55 smithi167 ceph-mon[37701]: pgmap v58: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:56.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2835616045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi167-75207-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:04:56.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4052815225' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi167-73991-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:04:56.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:55 smithi167 ceph-mon[37701]: osdmap e74: 8 total, 8 up, 8 in 2024-06-18T13:04:56.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:04:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/309801237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi167-75055-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:04:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:55 smithi167 ceph-mon[29496]: pgmap v58: 132 pgs: 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:04:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2835616045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi167-75207-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:04:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4052815225' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi167-73991-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:04:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:55 smithi167 ceph-mon[29496]: osdmap e74: 8 total, 8 up, 8 in 2024-06-18T13:04:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:04:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/309801237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi167-75055-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:04:58.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:04:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:05:01.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2299211644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi167-73976-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi167-75207-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi167-75055-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi167-73991-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:01.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2299211644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi167-73976-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:01.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:01.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi167-75207-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:01.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi167-75055-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:01.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi167-73991-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:01.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2299211644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi167-73976-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:01.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:01.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi167-75207-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:01.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi167-75055-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:01.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi167-73991-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:02.889 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-06-18T13:05:02.889 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [----------] Global test environment set-up. 2024-06-18T13:05:02.889 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [----------] 12 tests from AsioRados 2024-06-18T13:05:02.889 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-06-18T13:05:02.889 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (1 ms) 2024-06-18T13:05:02.890 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-06-18T13:05:02.890 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (0 ms) 2024-06-18T13:05:02.890 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-06-18T13:05:02.890 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (1 ms) 2024-06-18T13:05:02.890 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-06-18T13:05:02.890 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (2 ms) 2024-06-18T13:05:02.890 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-06-18T13:05:02.890 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (2 ms) 2024-06-18T13:05:02.890 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-06-18T13:05:02.890 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (2 ms) 2024-06-18T13:05:02.890 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-06-18T13:05:02.890 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (0 ms) 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (1 ms) 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (0 ms) 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (2 ms) 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (1 ms) 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (1 ms) 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [----------] 12 tests from AsioRados (13 ms total) 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [----------] Global test environment tear-down 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (8808 ms total) 2024-06-18T13:05:02.891 INFO:tasks.workunit.client.0.smithi167.stdout: api_asio: [ PASSED ] 12 tests. 2024-06-18T13:05:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: pgmap v60: 196 pgs: 64 unknown, 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:05:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1908496848' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi167-74123-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1584439026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi167-74023-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2644384632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi167-74007-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: pgmap v62: 356 pgs: 224 unknown, 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:05:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: pgmap v63: 356 pgs: 145 creating+peering, 53 unknown, 158 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:05:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: osdmap e75: 8 total, 8 up, 8 in 2024-06-18T13:05:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi167-74123-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi167-74023-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi167-74007-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2299211644' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi167-73976-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:03.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: pgmap v60: 196 pgs: 64 unknown, 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:05:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1908496848' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi167-74123-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1584439026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi167-74023-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2644384632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi167-74007-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: pgmap v62: 356 pgs: 224 unknown, 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:05:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: pgmap v63: 356 pgs: 145 creating+peering, 53 unknown, 158 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:05:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: osdmap e75: 8 total, 8 up, 8 in 2024-06-18T13:05:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi167-74123-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi167-74023-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi167-74007-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:03.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2299211644' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi167-73976-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: pgmap v60: 196 pgs: 64 unknown, 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:05:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1908496848' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi167-74123-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1584439026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi167-74023-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2644384632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi167-74007-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: pgmap v62: 356 pgs: 224 unknown, 132 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:05:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: pgmap v63: 356 pgs: 145 creating+peering, 53 unknown, 158 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:05:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: osdmap e75: 8 total, 8 up, 8 in 2024-06-18T13:05:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:03.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi167-74123-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi167-74023-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi167-74007-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:03.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:03.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2299211644' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi167-73976-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.357 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-06-18T13:05:04.357 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [----------] Global test environment set-up. 2024-06-18T13:05:04.357 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [----------] 7 tests from LibRadosList 2024-06-18T13:05:04.358 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-06-18T13:05:04.358 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ OK ] LibRadosList.ListObjects (363 ms) 2024-06-18T13:05:04.358 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-06-18T13:05:04.358 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (27 ms) 2024-06-18T13:05:04.358 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-06-18T13:05:04.358 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: myset foo1,foo2,foo3 2024-06-18T13:05:04.358 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo1 2024-06-18T13:05:04.358 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo2 2024-06-18T13:05:04.358 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo3 2024-06-18T13:05:04.358 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: myset foo1,foo4,foo5 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo4 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo5 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo1 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: myset foo6,foo7 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo7 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo6 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: ns1:foo4 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: ns1:foo5 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: ns2:foo7 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: ns2:foo6 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: ns1:foo1 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: :foo1 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: :foo2 2024-06-18T13:05:04.359 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: :foo3 2024-06-18T13:05:04.360 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (70 ms) 2024-06-18T13:05:04.360 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-06-18T13:05:04.360 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 1 0 2024-06-18T13:05:04.360 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 10 0 2024-06-18T13:05:04.360 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 13 0 2024-06-18T13:05:04.360 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 7 0 2024-06-18T13:05:04.360 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 14 0 2024-06-18T13:05:04.360 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 0 0 2024-06-18T13:05:04.360 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 15 0 2024-06-18T13:05:04.360 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 11 0 2024-06-18T13:05:04.360 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 5 0 2024-06-18T13:05:04.360 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 8 0 2024-06-18T13:05:04.360 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 6 0 2024-06-18T13:05:04.361 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 3 0 2024-06-18T13:05:04.361 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 4 0 2024-06-18T13:05:04.361 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 12 0 2024-06-18T13:05:04.361 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 9 0 2024-06-18T13:05:04.361 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 2 0 2024-06-18T13:05:04.361 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-06-18T13:05:04.361 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (67 ms) 2024-06-18T13:05:04.361 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-06-18T13:05:04.361 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: x cursor=MIN 2024-06-18T13:05:04.361 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=1 cursor=21:02547ec2:::1:head 2024-06-18T13:05:04.361 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=10 cursor=21:52ea6a34:::10:head 2024-06-18T13:05:04.361 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=13 cursor=21:566253c9:::13:head 2024-06-18T13:05:04.362 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=7 cursor=21:5c6b0b28:::7:head 2024-06-18T13:05:04.362 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=14 cursor=21:62a1935d:::14:head 2024-06-18T13:05:04.362 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=0 cursor=21:6cac518f:::0:head 2024-06-18T13:05:04.362 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=15 cursor=21:863748b0:::15:head 2024-06-18T13:05:04.362 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=11 cursor=21:89d3ae78:::11:head 2024-06-18T13:05:04.362 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=5 cursor=21:b29083e3:::5:head 2024-06-18T13:05:04.362 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=8 cursor=21:bd63b0f1:::8:head 2024-06-18T13:05:04.362 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=6 cursor=21:c4fdafeb:::6:head 2024-06-18T13:05:04.362 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=3 cursor=21:cfc208b3:::3:head 2024-06-18T13:05:04.362 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=4 cursor=21:d83876eb:::4:head 2024-06-18T13:05:04.362 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=12 cursor=21:de5d7c5f:::12:head 2024-06-18T13:05:04.362 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=9 cursor=21:e960b815:::9:head 2024-06-18T13:05:04.363 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > oid=2 cursor=21:f905c69b:::2:head 2024-06-18T13:05:04.363 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: FIRST> seek to MIN oid=1 2024-06-18T13:05:04.363 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=1 cursor=21:02547ec2:::1:head 2024-06-18T13:05:04.363 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:02547ec2:::1:head 2024-06-18T13:05:04.363 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:02547ec2:::1:head -> 1 2024-06-18T13:05:04.363 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=10 cursor=21:52ea6a34:::10:head 2024-06-18T13:05:04.363 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:52ea6a34:::10:head 2024-06-18T13:05:04.363 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:52ea6a34:::10:head -> 10 2024-06-18T13:05:04.363 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=13 cursor=21:566253c9:::13:head 2024-06-18T13:05:04.363 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:566253c9:::13:head 2024-06-18T13:05:04.363 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:566253c9:::13:head -> 13 2024-06-18T13:05:04.363 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=7 cursor=21:5c6b0b28:::7:head 2024-06-18T13:05:04.363 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:5c6b0b28:::7:head 2024-06-18T13:05:04.363 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:5c6b0b28:::7:head -> 7 2024-06-18T13:05:04.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=14 cursor=21:62a1935d:::14:head 2024-06-18T13:05:04.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:62a1935d:::14:head 2024-06-18T13:05:04.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:62a1935d:::14:head -> 14 2024-06-18T13:05:04.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=0 cursor=21:6cac518f:::0:head 2024-06-18T13:05:04.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:6cac518f:::0:head 2024-06-18T13:05:04.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:6cac518f:::0:head -> 0 2024-06-18T13:05:04.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=15 cursor=21:863748b0:::15:head 2024-06-18T13:05:04.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:863748b0:::15:head 2024-06-18T13:05:04.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:863748b0:::15:head -> 15 2024-06-18T13:05:04.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=11 cursor=21:89d3ae78:::11:head 2024-06-18T13:05:04.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:89d3ae78:::11:head 2024-06-18T13:05:04.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:89d3ae78:::11:head -> 11 2024-06-18T13:05:04.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=5 cursor=21:b29083e3:::5:head 2024-06-18T13:05:04.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:b29083e3:::5:head 2024-06-18T13:05:04.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:b29083e3:::5:head -> 5 2024-06-18T13:05:04.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=8 cursor=21:bd63b0f1:::8:head 2024-06-18T13:05:04.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:bd63b0f1:::8:head 2024-06-18T13:05:04.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:bd63b0f1:::8:head -> 8 2024-06-18T13:05:04.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=6 cursor=21:c4fdafeb:::6:head 2024-06-18T13:05:04.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:c4fdafeb:::6:head 2024-06-18T13:05:04.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:c4fdafeb:::6:head -> 6 2024-06-18T13:05:04.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=3 cursor=21:cfc208b3:::3:head 2024-06-18T13:05:04.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:cfc208b3:::3:head 2024-06-18T13:05:04.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:cfc208b3:::3:head -> 3 2024-06-18T13:05:04.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=4 cursor=21:d83876eb:::4:head 2024-06-18T13:05:04.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:d83876eb:::4:head 2024-06-18T13:05:04.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:d83876eb:::4:head -> 4 2024-06-18T13:05:04.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=12 cursor=21:de5d7c5f:::12:head 2024-06-18T13:05:04.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:de5d7c5f:::12:head 2024-06-18T13:05:04.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:de5d7c5f:::12:head -> 12 2024-06-18T13:05:04.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=9 cursor=21:e960b815:::9:head 2024-06-18T13:05:04.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:e960b815:::9:head 2024-06-18T13:05:04.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:e960b815:::9:head -> 9 2024-06-18T13:05:04.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : oid=2 cursor=21:f905c69b:::2:head 2024-06-18T13:05:04.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:f905c69b:::2:head 2024-06-18T13:05:04.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:f905c69b:::2:head -> 2 2024-06-18T13:05:04.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:c4fdafeb:::6:head 2024-06-18T13:05:04.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:c4fdafeb:::6:head expected=21:c4fdafeb:::6:head 2024-06-18T13:05:04.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:c4fdafeb:::6:head -> 6 2024-06-18T13:05:04.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=6 expected=6 2024-06-18T13:05:04.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:52ea6a34:::10:head 2024-06-18T13:05:04.454 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:52ea6a34:::10:head expected=21:52ea6a34:::10:head 2024-06-18T13:05:04.454 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:52ea6a34:::10:head -> 10 2024-06-18T13:05:04.454 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=10 expected=10 2024-06-18T13:05:04.454 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:e960b815:::9:head 2024-06-18T13:05:04.454 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:e960b815:::9:head expected=21:e960b815:::9:head 2024-06-18T13:05:04.454 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:e960b815:::9:head -> 9 2024-06-18T13:05:04.454 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=9 expected=9 2024-06-18T13:05:04.454 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:de5d7c5f:::12:head 2024-06-18T13:05:04.454 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:de5d7c5f:::12:head expected=21:de5d7c5f:::12:head 2024-06-18T13:05:04.454 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:de5d7c5f:::12:head -> 12 2024-06-18T13:05:04.454 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=12 expected=12 2024-06-18T13:05:04.454 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:f905c69b:::2:head 2024-06-18T13:05:04.454 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:f905c69b:::2:head expected=21:f905c69b:::2:head 2024-06-18T13:05:04.454 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:f905c69b:::2:head -> 2 2024-06-18T13:05:04.455 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=2 expected=2 2024-06-18T13:05:04.455 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:d83876eb:::4:head 2024-06-18T13:05:04.455 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:d83876eb:::4:head expected=21:d83876eb:::4:head 2024-06-18T13:05:04.455 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:d83876eb:::4:head -> 4 2024-06-18T13:05:04.455 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=4 expected=4 2024-06-18T13:05:04.455 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:6cac518f:::0:head 2024-06-18T13:05:04.455 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:6cac518f:::0:head expected=21:6cac518f:::0:head 2024-06-18T13:05:04.455 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:6cac518f:::0:head -> 0 2024-06-18T13:05:04.455 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=0 expected=0 2024-06-18T13:05:04.455 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:cfc208b3:::3:head 2024-06-18T13:05:04.455 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:cfc208b3:::3:head expected=21:cfc208b3:::3:head 2024-06-18T13:05:04.455 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:cfc208b3:::3:head -> 3 2024-06-18T13:05:04.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi167-75207-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:04.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi167-74075-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3344402788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4095166720' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi167-74410-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2342334906' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi167-74409-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3787369870' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi167-74069-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1275204173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi167-74374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2835616045' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi167-75207-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:04.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/979901754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi167-74266-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3273155186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi167-74332-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3696104644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74739-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1925702695' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3009775229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74585-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3477953162' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-18T13:05:04.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: pgmap v65: 1060 pgs: 145 creating+peering, 757 unknown, 158 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 755 B/s rd, 0 op/s 2024-06-18T13:05:04.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: osdmap e76: 8 total, 8 up, 8 in 2024-06-18T13:05:04.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:04.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi167-74075-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi167-75207-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:04.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1787305163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi167-74751-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi167-74410-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi167-74266-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi167-74409-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi167-74332-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi167-74069-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74739-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi167-74374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74585-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi167-74123-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi167-74023-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi167-74007-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi167-75055-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi167-73991-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: osdmap e77: 8 total, 8 up, 8 in 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/309801237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi167-75055-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/7593771' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi167-75055-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/787055667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi167-74459-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.640 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.641 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi167-74459-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.641 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1826116192' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-18T13:05:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi167-75207-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi167-74075-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3344402788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4095166720' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi167-74410-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2342334906' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi167-74409-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3787369870' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi167-74069-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1275204173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi167-74374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2835616045' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi167-75207-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/979901754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi167-74266-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3273155186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi167-74332-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3696104644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74739-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1925702695' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3009775229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74585-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3477953162' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-18T13:05:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: pgmap v65: 1060 pgs: 145 creating+peering, 757 unknown, 158 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 755 B/s rd, 0 op/s 2024-06-18T13:05:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: osdmap e76: 8 total, 8 up, 8 in 2024-06-18T13:05:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:04.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi167-75207-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:04.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi167-74075-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3344402788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4095166720' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi167-74410-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2342334906' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi167-74409-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3787369870' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi167-74069-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1275204173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi167-74374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2835616045' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi167-75207-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:04.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/979901754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi167-74266-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3273155186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi167-74332-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3696104644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74739-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1925702695' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3009775229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74585-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3477953162' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-18T13:05:04.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: pgmap v65: 1060 pgs: 145 creating+peering, 757 unknown, 158 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 755 B/s rd, 0 op/s 2024-06-18T13:05:04.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: osdmap e76: 8 total, 8 up, 8 in 2024-06-18T13:05:04.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:04.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi167-74075-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi167-75207-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:04.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1787305163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi167-74751-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi167-74410-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi167-74266-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi167-74409-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi167-74332-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi167-74069-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74739-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi167-74374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74585-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:04.731 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.731 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi167-74123-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.731 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi167-74023-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi167-74075-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi167-75207-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:04.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1787305163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi167-74751-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi167-74410-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi167-74266-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi167-74409-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi167-74332-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi167-74069-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74739-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi167-74374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74585-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:04.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi167-74123-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi167-74023-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.732 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi167-74007-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.732 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi167-75055-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:04.733 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi167-73991-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.733 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: osdmap e77: 8 total, 8 up, 8 in 2024-06-18T13:05:04.733 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/309801237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi167-75055-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:04.733 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/7593771' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.733 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi167-75055-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:04.733 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/787055667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi167-74459-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.733 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.733 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi167-74459-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.733 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1826116192' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-18T13:05:04.733 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi167-74007-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi167-75055-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:04.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi167-73991-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:04.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: osdmap e77: 8 total, 8 up, 8 in 2024-06-18T13:05:04.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/309801237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi167-75055-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:04.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/7593771' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi167-75055-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:04.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/787055667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi167-74459-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi167-74459-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:04.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1826116192' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-18T13:05:05.272 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=3 api_cmd_pp: Running main() from gmock_main.cc 2024-06-18T13:05:05.272 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-06-18T13:05:05.272 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-06-18T13:05:05.272 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-06-18T13:05:05.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-06-18T13:05:05.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (68 ms) 2024-06-18T13:05:05.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-06-18T13:05:05.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (64 ms) 2024-06-18T13:05:05.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-06-18T13:05:05.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (10891 ms) 2024-06-18T13:05:05.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (11023 ms total) 2024-06-18T13:05:05.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd_pp: 2024-06-18T13:05:05.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-06-18T13:05:05.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (11023 ms total) 2024-06-18T13:05:05.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-06-18T13:05:05.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-06-18T13:05:05.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-06-18T13:05:05.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-06-18T13:05:05.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-06-18T13:05:05.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (10956 ms) 2024-06-18T13:05:05.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (10956 ms total) 2024-06-18T13:05:05.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_cls_remote_reads: 2024-06-18T13:05:05.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-06-18T13:05:05.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (10956 ms total) 2024-06-18T13:05:05.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-06-18T13:05:05.275 INFO:tasks.workunit.client.0.smithi167.stdout: handler_error: Running main() from gmock_main.cc 2024-06-18T13:05:05.275 INFO:tasks.workunit.client.0.smithi167.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-06-18T13:05:05.275 INFO:tasks.workunit.client.0.smithi167.stdout: handler_error: [----------] Global test environment set-up. 2024-06-18T13:05:05.275 INFO:tasks.workunit.client.0.smithi167.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-06-18T13:05:05.275 INFO:tasks.workunit.client.0.smithi167.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-06-18T13:05:05.275 INFO:tasks.workunit.client.0.smithi167.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (10876 ms) 2024-06-18T13:05:05.275 INFO:tasks.workunit.client.0.smithi167.stdout: handler_error: [----------] 1 test from neocls_handler_error (10876 ms total) 2024-06-18T13:05:05.275 INFO:tasks.workunit.client.0.smithi167.stdout: handler_error: 2024-06-18T13:05:05.275 INFO:tasks.workunit.client.0.smithi167.stdout: handler_error: [----------] Global test environment tear-down 2024-06-18T13:05:05.275 INFO:tasks.workunit.client.0.smithi167.stdout: handler_error: [==========] 1 test from 1 test suite ran. (10876 ms total) 2024-06-18T13:05:05.275 INFO:tasks.workunit.client.0.smithi167.stdout: handler_error: [ PASSED ] 1 test. 2024-06-18T13:05:05.362 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_1_[74776]: starting. 2024-06-18T13:05:05.362 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_1_[74776]: creating pool ceph_test_rados_list_parallel.smithi167-74730 2024-06-18T13:05:05.362 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_1_[74776]: created object 0... 2024-06-18T13:05:05.362 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_1_[74776]: created object 25... 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_1_[74776]: created object 49... 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_1_[74776]: finishing. 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_1_[74776]: shutting down. 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_2_[74779]: starting. 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_2_[74779]: listing objects. 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_2_[74779]: listed object 0... 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_2_[74779]: listed object 25... 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_2_[74779]: saw 50 objects 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_2_[74779]: shutting down. 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_3_[75972]: starting. 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_3_[75972]: creating pool ceph_test_rados_list_parallel.smithi167-74730 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_3_[75972]: created object 0... 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_3_[75972]: created object 25... 2024-06-18T13:05:05.363 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_3_[75972]: created object 49... 2024-06-18T13:05:05.364 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_3_[75972]: finishing. 2024-06-18T13:05:05.364 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_3_[75972]: shutting down. 2024-06-18T13:05:05.364 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:05.364 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_4_[75973]: starting. 2024-06-18T13:05:05.364 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_4_[75973]: listing objects. 2024-06-18T13:05:05.364 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_4_[75973]: listed object 0... 2024-06-18T13:05:05.364 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_4_[75973]: listed object 25... 2024-06-18T13:05:05.364 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_4_[75973]: saw 50 objects 2024-06-18T13:05:05.364 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_4_[75973]: shutting down. 2024-06-18T13:05:05.364 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:05.364 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_5_[75974]: starting. 2024-06-18T13:05:05.364 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_5_[75974]: removed 25 objects... 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_5_[75974]: removed half of the objects 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_5_[75974]: removed 50 objects... 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_5_[75974]: removed 50 objects 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_5_[75974]: shutting down. 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_6_[76079]: starting. 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_6_[76079]: creating pool ceph_test_rados_list_parallel.smithi167-74730 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_6_[76079]: created object 0... 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_6_[76079]: created object 25... 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_6_[76079]: created object 49... 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_6_[76079]: finishing. 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_6_[76079]: shutting down. 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:05.365 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:05.366 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_7_[76080]: starting. 2024-06-18T13:05:05.366 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_7_[76080]: listing objects. 2024-06-18T13:05:05.366 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_7_[76080]: listed object 0... 2024-06-18T13:05:05.366 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_7_[76080]: listed object 25... 2024-06-18T13:05:05.366 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_7_[76080]: listed object 50... 2024-06-18T13:05:05.366 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_7_[76080]: saw 54 objects 2024-06-18T13:05:05.366 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_7_[76080]: shutting down. 2024-06-18T13:05:05.366 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:05.366 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:05.366 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_8_[76081]: starting. 2024-06-18T13:05:05.366 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_8_[76081]: added 25 objects... 2024-06-18T13:05:05.366 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_8_[76081]: added half of the objects 2024-06-18T13:05:05.366 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_8_[76081]: added 50 objects... 2024-06-18T13:05:05.367 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_8_[76081]: added 50 objects 2024-06-18T13:05:05.367 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_8_[76081]: shutting down. 2024-06-18T13:05:05.367 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:05.367 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:05.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:05:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:05:05.544 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ********************** api_c_read_operations: Running main() from gmock_main.cc 2024-06-18T13:05:05.544 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-06-18T13:05:05.544 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-06-18T13:05:05.544 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-06-18T13:05:05.544 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-06-18T13:05:05.544 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-06-18T13:05:05.544 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-06-18T13:05:05.544 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (18 ms) 2024-06-18T13:05:05.544 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (6 ms) 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (6 ms) 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (17 ms) 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (9 ms) 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (6 ms) 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (6 ms) 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (6 ms) 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-06-18T13:05:05.545 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (3 ms) 2024-06-18T13:05:05.546 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-06-18T13:05:05.546 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (3 ms) 2024-06-18T13:05:05.546 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-06-18T13:05:05.546 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (4 ms) 2024-06-18T13:05:05.546 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-06-18T13:05:05.546 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (5 ms) 2024-06-18T13:05:05.546 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-06-18T13:05:05.546 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (10 ms) 2024-06-18T13:05:05.546 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-06-18T13:05:05.546 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (11 ms) 2024-06-18T13:05:05.546 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-06-18T13:05:05.546 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (9 ms) 2024-06-18T13:05:05.547 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-06-18T13:05:05.547 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (5 ms) 2024-06-18T13:05:05.547 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (124 ms total) 2024-06-18T13:05:05.547 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: 2024-06-18T13:05:05.547 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-06-18T13:05:05.547 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (11232 ms total) 2024-06-18T13:05:05.547 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-06-18T13:05:05.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: Running main() from gmock_main.cc 2024-06-18T13:05:05.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-06-18T13:05:05.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [----------] Global test environment set-up. 2024-06-18T13:05:05.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-06-18T13:05:05.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-06-18T13:05:05.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (43 ms) 2024-06-18T13:05:05.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-06-18T13:05:05.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (60 ms) 2024-06-18T13:05:05.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-06-18T13:05:05.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (10948 ms) 2024-06-18T13:05:05.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-06-18T13:05:05.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.752221+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi167-74075-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.752327+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.752374+0000 mon.a [INF] from='client.? 172.21.15.167:0/1787305163' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi167-74751-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.752419+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi167-74410-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.752484+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi167-74266-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.752602+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi167-74409-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.752667+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi167-74332-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.752719+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi167-74069-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.752768+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74739-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.752821+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi167-74374-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.752889+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74504-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.752935+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74585-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.753002+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotif api_service: [==========] Running 4 tests from 1 test suite. 2024-06-18T13:05:05.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: [----------] Global test environment set-up. 2024-06-18T13:05:05.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: [----------] 4 tests from LibRadosService 2024-06-18T13:05:05.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-06-18T13:05:05.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5881 ms) 2024-06-18T13:05:05.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-06-18T13:05:05.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: [ OK ] LibRadosService.RegisterLate (15 ms) 2024-06-18T13:05:05.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-06-18T13:05:05.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: cluster: 2024-06-18T13:05:05.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: id: 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T13:05:05.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: health: HEALTH_WARN 2024-06-18T13:05:05.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2 pool(s) do not have an application enabled 2024-06-18T13:05:05.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:05.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: services: 2024-06-18T13:05:05.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-06-18T13:05:05.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: mgr: y(active, since 113s), standbys: x 2024-06-18T13:05:05.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-06-18T13:05:05.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-06-18T13:05:05.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-06-18T13:05:05.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:05.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: data: 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: pools: 8 pools, 196 pgs 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: objects: 198 objects, 583 KiB 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: usage: 243 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: pgs: 32.653% pgs unknown 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 132 active+clean 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 64 unknown 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: io: 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: client: 1023 B/s rd, 0 op/s rd, 0 op/s wr 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: cluster: 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: id: 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: health: HEALTH_WARN 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 7 pool(s) do not have an application enabled 2024-06-18T13:05:05.553 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: services: 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: mgr: y(active, since 115s), standbys: x 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: data: 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: pools: 13 pools, 356 pgs 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: objects: 198 objects, 583 KiB 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: usage: 244 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: pgs: 14.888% pgs unknown 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 40.730% pgs not active 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 158 active+clean 2024-06-18T13:05:05.554 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 145 creating+peering 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 53 unknown 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: io: 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: client: 1.2 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: cluster: 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: id: 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: health: HEALTH_WARN 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 30 pool(s) do not have an application enabled 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: services: 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: mgr: y(active, since 118s), standbys: x 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-06-18T13:05:05.555 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-06-18T13:05:05.556 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-06-18T13:05:05.556 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-06-18T13:05:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: pgmap v67: 1284 pgs: 826 creating+peering, 102 unknown, 356 active+clean; 587 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 8.4 KiB/s wr, 145 op/s 2024-06-18T13:05:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi167-74075-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1787305163' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi167-74751-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi167-74410-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi167-74266-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi167-74409-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi167-74332-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi167-74069-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74739-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi167-74374-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74504-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74585-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:05.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi167-74459-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi167-74507-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:05.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: osdmap e78: 8 total, 8 up, 8 in 2024-06-18T13:05:05.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi167-74507-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:05.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:05.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:05.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:05.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:05.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi167-74007-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: pgmap v67: 1284 pgs: 826 creating+peering, 102 unknown, 356 active+clean; 587 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 8.4 KiB/s wr, 145 op/s 2024-06-18T13:05:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi167-74075-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1787305163' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi167-74751-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi167-74410-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi167-74266-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi167-74409-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi167-74332-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi167-74069-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74739-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi167-74374-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74504-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74585-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:05.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi167-74459-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi167-74507-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:05.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: pgmap v67: 1284 pgs: 826 creating+peering, 102 unknown, 356 active+clean; 587 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 8.4 KiB/s wr, 145 op/s 2024-06-18T13:05:05.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi167-74075-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1787305163' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi167-74751-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi167-74410-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi167-74266-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi167-74409-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi167-74332-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi167-74069-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74739-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi167-74374-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74504-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74585-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:05.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi167-74459-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:05.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi167-74507-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:05.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: osdmap e78: 8 total, 8 up, 8 in 2024-06-18T13:05:05.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi167-74507-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:05.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:05.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:05.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:05.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:05.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi167-74007-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:05.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: osdmap e78: 8 total, 8 up, 8 in 2024-06-18T13:05:05.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi167-74507-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:05.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:05.981 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi167-74007-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:06.187 INFO:tasks.workunit.client.0.smithi167.stdout: ********* 2024-06-18T13:05:06.187 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_9_[76131]: starting. 2024-06-18T13:05:06.188 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_9_[76131]: creating pool ceph_test_rados_list_parallel.smithi167-74730 2024-06-18T13:05:06.188 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_9_[76131]: created object 0... 2024-06-18T13:05:06.188 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_9_[76131]: created object 25... 2024-06-18T13:05:06.188 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_9_[76131]: created object 49... 2024-06-18T13:05:06.188 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_9_[76131]: finishing. 2024-06-18T13:05:06.188 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_9_[76131]: shutting down. 2024-06-18T13:05:06.188 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.188 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.188 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.188 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_10_[76132]: starting. 2024-06-18T13:05:06.188 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_10_[76132]: listing objects. 2024-06-18T13:05:06.188 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_10_[76132]: listed object 0... 2024-06-18T13:05:06.188 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_10_[76132]: listed object 25... 2024-06-18T13:05:06.188 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_10_[76132]: listed object 50... 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_10_[76132]: listed object 75... 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_10_[76132]: saw 98 objects 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_10_[76132]: shutting down. 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_13_[76135]: starting. 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_13_[76135]: removed 25 objects... 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_13_[76135]: removed half of the objects 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_13_[76135]: removed 50 objects... 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_13_[76135]: removed 50 objects 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_13_[76135]: shutting down. 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.189 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.190 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_12_[76134]: starting. 2024-06-18T13:05:06.190 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_12_[76134]: added 25 objects... 2024-06-18T13:05:06.190 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_12_[76134]: added half of the objects 2024-06-18T13:05:06.190 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_12_[76134]: added 50 objects... 2024-06-18T13:05:06.190 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_12_[76134]: added 50 objects 2024-06-18T13:05:06.190 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_12_[76134]: shutting down. 2024-06-18T13:05:06.190 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.190 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.190 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.190 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_11_[76133]: starting. 2024-06-18T13:05:06.190 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_11_[76133]: added 25 objects... 2024-06-18T13:05:06.190 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_11_[76133]: added half of the objects 2024-06-18T13:05:06.190 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_11_[76133]: added 50 objects... 2024-06-18T13:05:06.190 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_11_[76133]: added 50 objects 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_11_[76133]: shutting down. 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_14_[76347]: starting. 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_14_[76347]: creating pool ceph_test_rados_list_parallel.smithi167-74730 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_14_[76347]: created object 0... 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_14_[76347]: created object 25... 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_14_[76347]: created object 49... 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_14_[76347]: finishing. 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_14_[76347]: shutting down. 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.191 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.192 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:06.192 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_15_[76348]: starting. 2024-06-18T13:05:06.192 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_15_[76348]: listing objects. 2024-06-18T13:05:06.192 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_15_[76348]: listed object 0... 2024-06-18T13:05:07.038 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_15_[76348]: listed object 2yECPP_smithi167-74507-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:07.038 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.753064+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:07.038 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.753125+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi167-74459-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:07.038 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:03.759092+0000 mon.c [INF] from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi167-74507-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:07.038 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:04.385896+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi167-74507-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:07.038 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:04.408295+0000 mon.b [INF] from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:07.038 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:04.666700+0000 mon.a [INF] from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:07.038 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:04.668920+0000 mon.a [INF] from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:07.038 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:04.669835+0000 mon.a [INF] from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi167-74007-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:07.039 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:04.956153+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi167-75207-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi167-75207-1"}]': finished 2024-06-18T13:05:07.039 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:04.956247+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi167-75055-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi167-75055-1"}]': finished 2024-06-18T13:05:07.039 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:04.956328+0000 mon.a [INF] from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi167-74007-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:07.039 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:05.434184+0000 mon.c [INF] from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-06-18T13:05:07.039 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:05.434765+0000 mon.c [INF] from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-06-18T13:05:07.039 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:05.435173+0000 mon.c [INF] from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-06-18T13:05:07.056 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:05.435703+0000 mon.c [INF] from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg5... 2024-06-18T13:05:07.057 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_15_[76348]: listed object 50... 2024-06-18T13:05:07.057 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_15_[76348]: listed object 75... 2024-06-18T13:05:07.057 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_15_[76348]: listed object 100... 2024-06-18T13:05:07.057 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_15_[76348]: listed object 125... 2024-06-18T13:05:07.057 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_15_[76348]: saw 150 objects 2024-06-18T13:05:07.057 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_15_[76348]: shutting down. 2024-06-18T13:05:07.057 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:07.057 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:07.057 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:07.057 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:07.057 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_16_[76349]: starting. 2024-06-18T13:05:07.057 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_16_[76349]: added 25 objects... 2024-06-18T13:05:07.058 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_16_[76349]: added half of the objects 2024-06-18T13:05:07.058 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_16_[76349]: added 50 objects... 2024-06-18T13:05:07.058 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_16_[76349]: added 50 objects 2024-06-18T13:05:07.058 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: process_16_[76349]: shutting down. 2024-06-18T13:05:07.058 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:07.058 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:07.058 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:07.058 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:07.058 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******************************* 2024-06-18T13:05:07.058 INFO:tasks.workunit.client.0.smithi167.stdout: list_parallel: ******* SUCCESS ********** 2024-06-18T13:05:07.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi167-75207-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi167-75207-1"}]': finished 2024-06-18T13:05:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi167-75055-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi167-75055-1"}]': finished 2024-06-18T13:05:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi167-74007-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-06-18T13:05:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-06-18T13:05:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-06-18T13:05:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-06-18T13:05:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-06-18T13:05:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-06-18T13:05:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-06-18T13:05:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-06-18T13:05:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-06-18T13:05:07.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-06-18T13:05:07.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: osdmap e79: 8 total, 8 up, 8 in 2024-06-18T13:05:07.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi167-74007-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:07.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/774256593' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-18T13:05:07.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:07.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1898361057' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-18T13:05:07.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi167-75207-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi167-75207-1"}]': finished 2024-06-18T13:05:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi167-75055-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi167-75055-1"}]': finished 2024-06-18T13:05:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi167-74007-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-06-18T13:05:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-06-18T13:05:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-06-18T13:05:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-06-18T13:05:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-06-18T13:05:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-06-18T13:05:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-06-18T13:05:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-06-18T13:05:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-06-18T13:05:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-06-18T13:05:07.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: osdmap e79: 8 total, 8 up, 8 in 2024-06-18T13:05:07.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi167-74007-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:07.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/774256593' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-18T13:05:07.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:07.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1898361057' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-18T13:05:07.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi167-75207-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi167-75207-1"}]': finished 2024-06-18T13:05:07.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi167-75055-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi167-75055-1"}]': finished 2024-06-18T13:05:07.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi167-74007-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:07.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-06-18T13:05:07.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-06-18T13:05:07.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-06-18T13:05:07.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-06-18T13:05:07.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-06-18T13:05:07.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-06-18T13:05:07.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-06-18T13:05:07.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-06-18T13:05:07.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-06-18T13:05:07.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-06-18T13:05:07.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: osdmap e79: 8 total, 8 up, 8 in 2024-06-18T13:05:07.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi167-74007-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:07.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/774256593' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-18T13:05:07.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:07.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1898361057' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-18T13:05:08.051 INFO:tasks.workunit.client.0.smithi167.stdout: deep-scrub", "pgid": "12.3"}]: dispatch 2024-06-18T13:05:08.051 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:05.436317+0000 mon.c [INF] from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-06-18T13:05:08.051 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:05.436936+0000 mon.c [INF] from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-06-18T13:05:08.051 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:05.437462+0000 mon.c [INF] from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-06-18T13:05:08.051 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:05.438000+0000 mon.c [INF] from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-06-18T13:05:08.051 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:05.438543+0000 mon.c [INF] from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-06-18T13:05:08.051 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:05.439058+0000 mon.c [INF] from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-06-18T13:05:08.051 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:05.540794+0000 mon.a [INF] from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi167-74007-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:08.051 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:05.549517+0000 mon.b [INF] from='client.? 172.21.15.167:0/774256593' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-18T13:05:08.051 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:06.042390+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi167-74507-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]': finished 2024-06-18T13:05:08.051 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:06.053701+0000 mon.c [INF] from='client.? 172.21.15.167:0/309801237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:08.052 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:06.054475+0000 mon.c [INF] from='client.? 172.21.15.167:0/1034922069' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi167-73976-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:08.052 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:06.556538+0000 mon.a [WRN] Health check update: 30 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:08.052 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:06.560456+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:08.052 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:06.560944+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi167-73976-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:08.052 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:06.562632+0000 mon.a [INF] from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:08.052 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:06.800397+0000 mon.a [INF] from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-06-18T13:05:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-06-18T13:05:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-06-18T13:05:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-06-18T13:05:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-06-18T13:05:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-06-18T13:05:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-06-18T13:05:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-06-18T13:05:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-06-18T13:05:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-06-18T13:05:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: pgmap v70: 852 pgs: 442 creating+peering, 182 unknown, 228 active+clean; 584 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 4.8 KiB/s wr, 226 op/s 2024-06-18T13:05:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: 12.3 deep-scrub starts 2024-06-18T13:05:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: 12.3 deep-scrub ok 2024-06-18T13:05:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: 12.8 deep-scrub starts 2024-06-18T13:05:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi167-74507-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]': finished 2024-06-18T13:05:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: 12.7 deep-scrub starts 2024-06-18T13:05:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/309801237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1034922069' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi167-73976-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: 12.7 deep-scrub ok 2024-06-18T13:05:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: 12.8 deep-scrub ok 2024-06-18T13:05:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: osdmap e80: 8 total, 8 up, 8 in 2024-06-18T13:05:08.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: Health check update: 30 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:08.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:08.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi167-73976-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:08.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:08.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:08.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:08.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi167-74007-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi167-74007-16"}]': finished 2024-06-18T13:05:08.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi167-75055-1"}]': finished 2024-06-18T13:05:08.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi167-73976-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:08.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:08.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/309801237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:08.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: osdmap e81: 8 total, 8 up, 8 in 2024-06-18T13:05:08.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:08.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi167-74123-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:08.390 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:08.390 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:08.390 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:08.390 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:08.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-06-18T13:05:08.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-06-18T13:05:08.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-06-18T13:05:08.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-06-18T13:05:08.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-06-18T13:05:08.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-06-18T13:05:08.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-06-18T13:05:08.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-06-18T13:05:08.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-06-18T13:05:08.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-06-18T13:05:08.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: pgmap v70: 852 pgs: 442 creating+peering, 182 unknown, 228 active+clean; 584 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 4.8 KiB/s wr, 226 op/s 2024-06-18T13:05:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: 12.3 deep-scrub starts 2024-06-18T13:05:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: 12.3 deep-scrub ok 2024-06-18T13:05:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: 12.8 deep-scrub starts 2024-06-18T13:05:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi167-74507-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]': finished 2024-06-18T13:05:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: 12.7 deep-scrub starts 2024-06-18T13:05:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/309801237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1034922069' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi167-73976-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: 12.7 deep-scrub ok 2024-06-18T13:05:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: 12.8 deep-scrub ok 2024-06-18T13:05:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: osdmap e80: 8 total, 8 up, 8 in 2024-06-18T13:05:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: Health check update: 30 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi167-73976-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi167-74007-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi167-74007-16"}]': finished 2024-06-18T13:05:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi167-75055-1"}]': finished 2024-06-18T13:05:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi167-73976-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/309801237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: osdmap e81: 8 total, 8 up, 8 in 2024-06-18T13:05:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi167-74123-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:08.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:08.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-06-18T13:05:08.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-06-18T13:05:08.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-06-18T13:05:08.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-06-18T13:05:08.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-06-18T13:05:08.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-06-18T13:05:08.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-06-18T13:05:08.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-06-18T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-06-18T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-06-18T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: pgmap v70: 852 pgs: 442 creating+peering, 182 unknown, 228 active+clean; 584 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 4.8 KiB/s wr, 226 op/s 2024-06-18T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: 12.3 deep-scrub starts 2024-06-18T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: 12.3 deep-scrub ok 2024-06-18T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: 12.8 deep-scrub starts 2024-06-18T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi167-74507-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]': finished 2024-06-18T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: 12.7 deep-scrub starts 2024-06-18T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/309801237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1034922069' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi167-73976-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: 12.7 deep-scrub ok 2024-06-18T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: 12.8 deep-scrub ok 2024-06-18T13:05:08.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: osdmap e80: 8 total, 8 up, 8 in 2024-06-18T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: Health check update: 30 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi167-73976-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi167-74007-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi167-74007-16"}]': finished 2024-06-18T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi167-75055-1"}]': finished 2024-06-18T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi167-73976-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/309801237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:08.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: osdmap e81: 8 total, 8 up, 8 in 2024-06-18T13:05:08.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:08.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi167-74123-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:08.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:08.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:08.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:08.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:08.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:05:08 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:05:09.070 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:07.037490+0000 mon.a [INF] from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.071 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:07.045611+0000 mon.a [INF] from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi167-74007-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi167-74007-16"}]': finished 2024-06-18T13:05:09.071 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:07.045672+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi167-75055-1"}]': finished 2024-06-18T13:05:09.071 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:07.045732+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi167-73976-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:09.071 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:07.045800+0000 mon.a [INF] from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:09.071 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:07.055324+0000 mon.c [INF] from='client.? 172.21.15.167:0/309801237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:09.071 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:07.065440+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi167-75055-1"}]: dispatch 2024-06-18T13:05:09.071 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:07.069111+0000 mon.a [INF] from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi167-74123-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:09.071 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:07.805942+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:09.071 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:07.806937+0000 mon.b [INF] from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:09.071 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:07.810151+0000 mon.a [INF] from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:09.071 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:05.549650+0000 client.admin [INF] onexx 2024-06-18T13:05:09.071 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.057031+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi167-75055-1"}]': finished 2024-06-18T13:05:09.071 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.110079+0000 mon.c [INF] from='client.? 172.21.15.167:0/626807822' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi167-75055-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.071 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.333125+0000 mon.b [INF] from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:09.072 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.334392+0000 mon.b [INF] from='client.? 172.21.15.167:0/1351959733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:09.072 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.337321+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi167-75055-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: onexx 2024-06-18T13:05:09.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: 12.1 deep-scrub starts 2024-06-18T13:05:09.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: 12.1 deep-scrub ok 2024-06-18T13:05:09.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: 12.5 deep-scrub starts 2024-06-18T13:05:09.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: 12.5 deep-scrub ok 2024-06-18T13:05:09.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: pgmap v73: 892 pgs: 378 creating+peering, 318 unknown, 196 active+clean; 584 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:05:09.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:05:09.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi167-75055-1"}]': finished 2024-06-18T13:05:09.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/626807822' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi167-75055-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:09.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1351959733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:09.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: osdmap e82: 8 total, 8 up, 8 in 2024-06-18T13:05:09.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi167-75055-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:09.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:09.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-18T13:05:09.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:09.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:09.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:09.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:09.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:09.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:09.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:09.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: onexx 2024-06-18T13:05:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: 12.1 deep-scrub starts 2024-06-18T13:05:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: 12.1 deep-scrub ok 2024-06-18T13:05:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: 12.5 deep-scrub starts 2024-06-18T13:05:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: 12.5 deep-scrub ok 2024-06-18T13:05:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: pgmap v73: 892 pgs: 378 creating+peering, 318 unknown, 196 active+clean; 584 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:05:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:05:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi167-75055-1"}]': finished 2024-06-18T13:05:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/626807822' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi167-75055-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1351959733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: osdmap e82: 8 total, 8 up, 8 in 2024-06-18T13:05:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi167-75055-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:09.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:09.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-18T13:05:09.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:09.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:09.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:09.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:09.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:09.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: onexx 2024-06-18T13:05:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: 12.1 deep-scrub starts 2024-06-18T13:05:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: 12.1 deep-scrub ok 2024-06-18T13:05:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: 12.5 deep-scrub starts 2024-06-18T13:05:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: 12.5 deep-scrub ok 2024-06-18T13:05:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: pgmap v73: 892 pgs: 378 creating+peering, 318 unknown, 196 active+clean; 584 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:05:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:05:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi167-75055-1"}]': finished 2024-06-18T13:05:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/626807822' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi167-75055-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1351959733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: osdmap e82: 8 total, 8 up, 8 in 2024-06-18T13:05:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi167-75055-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-18T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:09.481 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:09.481 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.481 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:09.481 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:09.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:09.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:09.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:09.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi167-74410-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-06-18T13:05:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/774256593' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-06-18T13:05:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: twoxx 2024-06-18T13:05:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-06-18T13:05:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi167-74123-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi167-74123-10"}]': finished 2024-06-18T13:05:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi167-75055-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:10.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:10.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:10.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:10.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/626807822' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi167-75055-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:10.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi167-74409-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:10.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1300444662' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:10.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2835616045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:10.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:10.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:10.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: osdmap e83: 8 total, 8 up, 8 in 2024-06-18T13:05:10.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:10.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi167-75055-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:10.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi167-74410-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-06-18T13:05:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/774256593' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-06-18T13:05:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: twoxx 2024-06-18T13:05:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-06-18T13:05:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi167-74123-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi167-74123-10"}]': finished 2024-06-18T13:05:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi167-75055-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/626807822' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi167-75055-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi167-74409-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1300444662' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2835616045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: osdmap e83: 8 total, 8 up, 8 in 2024-06-18T13:05:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi167-75055-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:10.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi167-74410-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:10.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-06-18T13:05:10.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/774256593' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-06-18T13:05:10.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: twoxx 2024-06-18T13:05:10.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi167-74123-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi167-74123-10"}]': finished 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi167-75055-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/626807822' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi167-75055-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi167-74409-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1300444662' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2835616045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: osdmap e83: 8 total, 8 up, 8 in 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:10.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi167-75055-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:11.608 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.374189+0000 mon.c [INF] from='client.? 1 watch_notify: Running main() from gmock_main.cc 2024-06-18T13:05:11.608 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-06-18T13:05:11.608 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: [----------] Global test environment set-up. 2024-06-18T13:05:11.608 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-06-18T13:05:11.608 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-06-18T13:05:11.608 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: handle_notify cookie 94538189281664 notify_id 335007449089 notifier_gid 24833 2024-06-18T13:05:11.608 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (10768 ms) 2024-06-18T13:05:11.608 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-06-18T13:05:11.608 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: Trying... 2024-06-18T13:05:11.608 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: handle_notify cookie 94538201982208 notify_id 347892350976 notifier_gid 25088 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: Waiting for 3.000000000s 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: Timed out. 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: Flushing... 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: Flushed... 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (6335 ms) 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (17103 ms total) 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: [----------] Global test environment tear-down 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (17103 ms total) 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: watch_notify: [ PASSED ] 2 tests. 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: cls: Running main() from gmock_main.cc 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: cls: [----------] Global test environment set-up. 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-06-18T13:05:11.609 INFO:tasks.workunit.client.0.smithi167.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-06-18T13:05:11.610 INFO:tasks.workunit.client.0.smithi167.stdout: cls: [ OK ] NeoRadosCls.DNE (11206 ms) 2024-06-18T13:05:11.610 INFO:tasks.workunit.client.0.smithi167.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-06-18T13:05:11.610 INFO:tasks.workunit.client.0.smithi167.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (6065 ms) 2024-06-18T13:05:11.610 INFO:tasks.workunit.client.0.smithi167.stdout: cls: [----------] 2 tests from NeoRadosCls (17271 ms total) 2024-06-18T13:05:11.610 INFO:tasks.workunit.client.0.smithi167.stdout: cls: 2024-06-18T13:05:11.610 INFO:tasks.workunit.client.0.smithi167.stdout: cls: [----------] Global test environment tear-down 2024-06-18T13:05:11.610 INFO:tasks.workunit.client.0.smithi167.stdout: cls: [==========] 2 tests from 1 test suite ran. (17271 ms total) 2024-06-18T13:05:11.610 INFO:tasks.workunit.client.0.smithi167.stdout: cls: [ PASSED ] 2 tests. 2024-06-18T13:05:11.610 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_1_[74998]: starting. 2024-06-18T13:05:11.611 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_1_[74998]: creating pool ceph_test_rados_open_pools_parallel.smithi167-74891 2024-06-18T13:05:11.611 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_1_[74998]: created object 0... 2024-06-18T13:05:11.611 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_1_[74998]: created object 25... 2024-06-18T13:05:11.611 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_1_[74998]: created object 49... 2024-06-18T13:05:11.611 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_1_[74998]: finishing. 2024-06-18T13:05:11.611 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_1_[74998]: shutting down. 2024-06-18T13:05:11.611 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_2_[75008]: starting. 2024-06-18T13:05:11.611 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_2_[75008]: rados_pool_create. 2024-06-18T13:05:11.611 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_2_[75008]: rados_ioctx_create. 2024-06-18T13:05:11.611 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_2_[75008]: shutting down. 2024-06-18T13:05:11.611 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: ******************************* 2024-06-18T13:05:11.611 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_3_[76220]: starting. 2024-06-18T13:05:11.611 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_3_[76220]: creating pool ceph_test_rados_open_pools_parallel.smithi167-74891 2024-06-18T13:05:11.611 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_3_[76220]: created object 0... 2024-06-18T13:05:11.612 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_3_[76220]: created object 25... 2024-06-18T13:05:11.612 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_3_[76220]: created object 49... 2024-06-18T13:05:11.612 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_3_[76220]: finishing. 2024-06-18T13:05:11.612 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_3_[76220]: shutting down. 2024-06-18T13:05:11.612 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: ******************************* 2024-06-18T13:05:11.612 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_4_[76221]: starting. 2024-06-18T13:05:11.612 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_4_[76221]: rados_pool_create. 2024-06-18T13:05:11.612 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_4_[76221]: rados_ioctx_create. 2024-06-18T13:05:11.612 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: process_4_[76221]: shutting down. 2024-06-18T13:05:11.612 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: ******************************* 2024-06-18T13:05:11.612 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: ******************************* 2024-06-18T13:05:11.612 INFO:tasks.workunit.client.0.smithi167.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-06-18T13:05:11.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: 12.4 deep-scrub starts 2024-06-18T13:05:11.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: 12.4 deep-scrub ok 2024-06-18T13:05:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: pgmap v76: 796 pgs: 191 creating+peering, 104 unknown, 501 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 301 op/s 2024-06-18T13:05:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi167-74409-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-06-18T13:05:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi167-74410-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi167-74023-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1259212712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1351959733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: osdmap e84: 8 total, 8 up, 8 in 2024-06-18T13:05:11.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:11.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:11.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi167-74410-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:11.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:11.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:11.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi167-74069-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:11.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi167-74069-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:11.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi167-74023-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:11.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: 12.4 deep-scrub starts 2024-06-18T13:05:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: 12.4 deep-scrub ok 2024-06-18T13:05:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: pgmap v76: 796 pgs: 191 creating+peering, 104 unknown, 501 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 301 op/s 2024-06-18T13:05:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi167-74409-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-06-18T13:05:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi167-74410-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi167-74023-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1259212712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1351959733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: osdmap e84: 8 total, 8 up, 8 in 2024-06-18T13:05:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi167-74410-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi167-74069-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:11.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi167-74069-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:11.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi167-74023-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:11.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: 12.4 deep-scrub starts 2024-06-18T13:05:11.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: 12.4 deep-scrub ok 2024-06-18T13:05:11.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:11.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: pgmap v76: 796 pgs: 191 creating+peering, 104 unknown, 501 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 301 op/s 2024-06-18T13:05:11.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi167-74409-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:11.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-06-18T13:05:11.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:11.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi167-74410-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:11.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:11.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:11.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi167-74023-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:11.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1259212712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:11.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1351959733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: osdmap e84: 8 total, 8 up, 8 in 2024-06-18T13:05:11.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:11.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:11.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi167-74410-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:11.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:11.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:11.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi167-74069-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:11.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi167-74069-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:11.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi167-74023-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:11.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:11.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:12.312 INFO:tasks.workunit.client.0.smithi167.stdout:72.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:12.312 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.596966+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-18T13:05:12.312 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.829705+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:12.312 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.830398+0000 mon.c [INF] from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:12.312 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.831092+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:12.312 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.831354+0000 mon.a [INF] from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:12.312 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.831652+0000 mon.c [INF] from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:12.313 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.832076+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:12.313 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.832536+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:12.313 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.833273+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:12.313 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.833430+0000 mon.b [INF] from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:12.313 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.833570+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:12.313 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:08.833822+0000 mon.a [INF] from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:12.313 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:10.075659+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi167-74409-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:12.313 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:10.308415+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-06-18T13:05:12.313 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:10.316042+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:12.313 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:10.319550+0000 mon.a [INF] from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi167-74410-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:12.313 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:10.319633+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:12.313 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:10.319712+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:12.313 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:10.325172+0000 mon.b [INF] from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi167-74023-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:12.314 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:10.326616+0000 mon.b [INF] from='client.? 172.21.15.167:0/1259212712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:12.314 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:10.570098+0000 mon.b [INF] from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:12.314 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:10.803209+0000 mon.b [INF] from='client.? 172.21.15.167:0/1351959733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:12.314 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.040093+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:12.314 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.040760+0000 mon.c [INF] from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:12.314 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.041037+0000 mon.a [INF] from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi167-74410-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:12.314 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.278124+0000 mon.a [INF] from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:12.314 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.280186+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:12.314 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.280688+0000 mon.c [INF] from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi167-74069-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:12.314 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.281006+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi167-74069-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:12.314 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.285398+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi167-74023-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:12.353 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.285606+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:12.353 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.286001+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:12.353 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.286143+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:12.353 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.322984+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi167-75055-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi167-75055-2"}]': finished 2024-06-18T13:05:12.353 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.323040+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi167-74409-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi167-74409-7"}]': finished 2024-06-18T13:05:12.353 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.323079+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi167-75207-1"}]': finished 2024-06-18T13:05:12.353 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.323110+0000 mon.a [INF] from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi167-74007-16"}]': finished 2024-06-18T13:05:12.353 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.323141+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi167-74069-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:12.353 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.323184+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:12.353 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.323259+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:12.354 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.323338+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:12.354 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.328947+0000 mon.c [INF] from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi167-74069-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:12.354 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.332738+0000 mon.b [INF] from='client.? 172.21.15.167:0/1351959733' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi167-73991-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-18T13:05:12.354 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.333107+0000 mon.b [INF] from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:12.354 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.333479+0000 mon.c [INF] from='client.? 172.21.15.167:0/2835616045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: 12.9 deep-scrub starts 2024-06-18T13:05:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: 12.9 deep-scrub ok 2024-06-18T13:05:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi167-75055-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi167-75055-2"}]': finished 2024-06-18T13:05:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi167-74409-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi167-74409-7"}]': finished 2024-06-18T13:05:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi167-75207-1"}]': finished 2024-06-18T13:05:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi167-74007-16"}]': finished 2024-06-18T13:05:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi167-74069-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi167-74069-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1351959733' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi167-73991-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-18T13:05:12.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:12.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2835616045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:12.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:12.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:12.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: osdmap e85: 8 total, 8 up, 8 in 2024-06-18T13:05:12.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:12.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:12.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi167-73991-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-18T13:05:12.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi167-74069-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:12.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:12.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:12.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi167-74410-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi167-74410-7"}]': finished 2024-06-18T13:05:12.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi167-74023-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi167-74023-22"}]': finished 2024-06-18T13:05:12.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: osdmap e86: 8 total, 8 up, 8 in 2024-06-18T13:05:12.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:12.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: 12.9 deep-scrub starts 2024-06-18T13:05:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: 12.9 deep-scrub ok 2024-06-18T13:05:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi167-75055-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi167-75055-2"}]': finished 2024-06-18T13:05:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi167-74409-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi167-74409-7"}]': finished 2024-06-18T13:05:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi167-75207-1"}]': finished 2024-06-18T13:05:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi167-74007-16"}]': finished 2024-06-18T13:05:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi167-74069-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi167-74069-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:12.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1351959733' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi167-73991-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-18T13:05:12.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:12.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2835616045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:12.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:12.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:12.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: osdmap e85: 8 total, 8 up, 8 in 2024-06-18T13:05:12.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:12.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:12.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi167-73991-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-18T13:05:12.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: 12.9 deep-scrub starts 2024-06-18T13:05:12.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: 12.9 deep-scrub ok 2024-06-18T13:05:12.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi167-75055-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi167-75055-2"}]': finished 2024-06-18T13:05:12.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi167-74409-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi167-74409-7"}]': finished 2024-06-18T13:05:12.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi167-75207-1"}]': finished 2024-06-18T13:05:12.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi167-74007-16"}]': finished 2024-06-18T13:05:12.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi167-74069-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:12.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:12.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:12.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi167-73991-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:12.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi167-74069-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:12.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1351959733' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi167-73991-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-18T13:05:12.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:12.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2835616045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:12.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:12.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:12.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: osdmap e85: 8 total, 8 up, 8 in 2024-06-18T13:05:12.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:12.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:12.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi167-73991-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-18T13:05:12.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi167-74069-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:12.731 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:12.731 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:12.731 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi167-74410-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi167-74410-7"}]': finished 2024-06-18T13:05:12.731 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi167-74023-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi167-74023-22"}]': finished 2024-06-18T13:05:12.731 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: osdmap e86: 8 total, 8 up, 8 in 2024-06-18T13:05:12.731 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:12.731 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:12.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi167-74069-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:12.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:12.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:12.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi167-74410-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi167-74410-7"}]': finished 2024-06-18T13:05:12.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi167-74023-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi167-74023-22"}]': finished 2024-06-18T13:05:12.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: osdmap e86: 8 total, 8 up, 8 in 2024-06-18T13:05:12.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:12.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:12.982 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T api_io_pp: Running main() from gmock_main.cc 2024-06-18T13:05:12.982 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-06-18T13:05:12.982 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [----------] Global test environment set-up. 2024-06-18T13:05:12.982 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-06-18T13:05:12.982 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: seed 74023 2024-06-18T13:05:12.982 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-06-18T13:05:12.982 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-06-18T13:05:12.982 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-06-18T13:05:12.982 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (98 ms) 2024-06-18T13:05:12.982 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-06-18T13:05:12.982 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (12 ms) 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (9 ms) 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (2 ms) 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (2 ms) 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (2 ms) 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (1 ms) 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (4 ms) 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (2 ms) 2024-06-18T13:05:12.983 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (3 ms) 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (4 ms) 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (3 ms) 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (3 ms) 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (3 ms) 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (8 ms) 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (4 ms) 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (2 ms) 2024-06-18T13:05:12.984 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (3 ms) 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (3 ms) 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (168 ms total) 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1090 ms) 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (12 ms) 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (5 ms) 2024-06-18T13:05:12.985 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-06-18T13:05:12.986 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (2 ms) 2024-06-18T13:05:13.147 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] L api_io: Running main() from gmock_main.cc 2024-06-18T13:05:13.147 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-06-18T13:05:13.147 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [----------] Global test environment set-up. 2024-06-18T13:05:13.147 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (120 ms) 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: no timeout :/ 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: no timeout :/ 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: no timeout :/ 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: no timeout :/ 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: no timeout :/ 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (33 ms) 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (7 ms) 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-06-18T13:05:13.148 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIo.Checksum (3 ms) 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (5 ms) 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (3 ms) 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (5 ms) 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (1 ms) 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIo.TruncTest (4 ms) 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (3 ms) 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (2 ms) 2024-06-18T13:05:13.149 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-06-18T13:05:13.150 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIo.RmXattr (8 ms) 2024-06-18T13:05:13.150 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-06-18T13:05:13.150 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIo.XattrIter (3 ms) 2024-06-18T13:05:13.150 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [----------] 14 tests from LibRadosIo (197 ms total) 2024-06-18T13:05:13.150 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: 2024-06-18T13:05:13.150 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-06-18T13:05:13.150 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-06-18T13:05:13.150 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (2133 ms) 2024-06-18T13:05:13.150 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-06-18T13:05:13.150 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (7 ms) 2024-06-18T13:05:13.150 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-06-18T13:05:13.150 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (7 ms) 2024-06-18T13:05:13.150 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-06-18T13:05:13.150 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (5 ms) 2024-06-18T13:05:13.151 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-06-18T13:05:13.151 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (9 ms) 2024-06-18T13:05:13.151 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-06-18T13:05:13.151 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (5 ms) 2024-06-18T13:05:13.151 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-06-18T13:05:13.151 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (4 ms) 2024-06-18T13:05:13.151 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-06-18T13:05:13.151 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-06-18T13:05:13.151 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-06-18T13:05:13.151 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-06-18T13:05:13.151 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-06-18T13:05:13.151 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (4 ms) 2024-06-18T13:05:13.151 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [----------] 10 tests from LibRadosIoEC (2186 ms total) 2024-06-18T13:05:13.151 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: 2024-06-18T13:05:13.152 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [----------] Global test environment tear-down 2024-06-18T13:05:13.152 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [==========] 24 tests from 2 test suites ran. (19108 ms total) 2024-06-18T13:05:13.152 INFO:tasks.workunit.client.0.smithi167.stdout: api_io: [ PASSED ] 24 tests. 2024-06-18T13:05:13.882 INFO:tasks.workunit.client.0.smithi167.stdout:13:05:11.605772+0000 mon.c [INF] from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:13.882 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.840990+0000 mon.a [WRN] Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:13.882 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.846577+0000 mon.a [INF] from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi167-74007-16"}]: dispatch 2024-06-18T13:05:13.882 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.848073+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi167-73991-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-18T13:05:13.882 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.848304+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi167-74069-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:13.883 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.848551+0000 mon.a [INF] from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:13.883 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.848816+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:13.883 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.849221+0000 mon.a [INF] from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi167-74410-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi167-74410-7"}]': finished 2024-06-18T13:05:13.883 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.849291+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi167-74023-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi167-74023-22"}]': finished 2024-06-18T13:05:13.883 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:11.856836+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi167-75207-1"}]: dispatch 2024-06-18T13:05:13.883 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:12.119301+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:13.883 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:12.313106+0000 mon.b [INF] from='client.? 172.21.15.167:0/774256593' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-18T13:05:13.883 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:12.313305+0000 client.admin [INF] threexx 2024-06-18T13:05:13.883 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:12.836617+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-18T13:05:13.883 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:12.853298+0000 mon.a [INF] from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi167-74007-16"}]': finished 2024-06-18T13:05:13.883 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:12.853354+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi167-73991-3", "field": "max_bytes", "val": "4096"}]': finished 2024-06-18T13:05:13.883 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:12.853454+0000 mon.a [INF] from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:13.907 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: go cmd: Running main() from gmock_main.cc 2024-06-18T13:05:13.907 INFO:tasks.workunit.client.0.smithi167.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-06-18T13:05:13.907 INFO:tasks.workunit.client.0.smithi167.stdout: cmd: [----------] Global test environment set-up. 2024-06-18T13:05:13.907 INFO:tasks.workunit.client.0.smithi167.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-06-18T13:05:13.907 INFO:tasks.workunit.client.0.smithi167.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-06-18T13:05:13.907 INFO:tasks.workunit.client.0.smithi167.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (9699 ms) 2024-06-18T13:05:13.907 INFO:tasks.workunit.client.0.smithi167.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-06-18T13:05:13.907 INFO:tasks.workunit.client.0.smithi167.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (5000 ms) 2024-06-18T13:05:13.907 INFO:tasks.workunit.client.0.smithi167.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-06-18T13:05:13.907 INFO:tasks.workunit.client.0.smithi167.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (4813 ms) 2024-06-18T13:05:13.907 INFO:tasks.workunit.client.0.smithi167.stdout: cmd: [----------] 3 tests from NeoRadosCmd (19512 ms total) 2024-06-18T13:05:13.907 INFO:tasks.workunit.client.0.smithi167.stdout: cmd: 2024-06-18T13:05:13.907 INFO:tasks.workunit.client.0.smithi167.stdout: cmd: [----------] Global test environment tear-down 2024-06-18T13:05:13.907 INFO:tasks.workunit.client.0.smithi167.stdout: cmd: [==========] 3 tests from 1 test suite ran. (19513 ms total) 2024-06-18T13:05:13.908 INFO:tasks.workunit.client.0.smithi167.stdout: cmd: [ PASSED ] 3 tests. 2024-06-18T13:05:14.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: 12.0 deep-scrub starts 2024-06-18T13:05:14.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: 12.0 deep-scrub ok 2024-06-18T13:05:14.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: pgmap v79: 796 pgs: 96 creating+peering, 280 unknown, 420 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 301 op/s 2024-06-18T13:05:14.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: 12.6 deep-scrub starts 2024-06-18T13:05:14.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: 12.6 deep-scrub ok 2024-06-18T13:05:14.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: 12.2 deep-scrub starts 2024-06-18T13:05:14.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: 12.2 deep-scrub ok 2024-06-18T13:05:14.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/774256593' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-18T13:05:14.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: threexx 2024-06-18T13:05:14.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:14.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-18T13:05:14.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi167-74007-16"}]': finished 2024-06-18T13:05:14.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi167-73991-3", "field": "max_bytes", "val": "4096"}]': finished 2024-06-18T13:05:14.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:14.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]': finished 2024-06-18T13:05:14.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi167-75207-1"}]': finished 2024-06-18T13:05:14.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]': finished 2024-06-18T13:05:14.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-4", "overlaypool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:14.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:14.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1683915713' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi167-75207-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:14.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: osdmap e87: 8 total, 8 up, 8 in 2024-06-18T13:05:14.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3841079011' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:14.139 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-4", "overlaypool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:14.139 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:14.139 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4191097629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi167-73976-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:14.139 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi167-75207-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:14.139 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: 12.0 deep-scrub starts 2024-06-18T13:05:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: 12.0 deep-scrub ok 2024-06-18T13:05:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: pgmap v79: 796 pgs: 96 creating+peering, 280 unknown, 420 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 301 op/s 2024-06-18T13:05:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: 12.6 deep-scrub starts 2024-06-18T13:05:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: 12.6 deep-scrub ok 2024-06-18T13:05:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: 12.2 deep-scrub starts 2024-06-18T13:05:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: 12.2 deep-scrub ok 2024-06-18T13:05:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/774256593' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-18T13:05:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: threexx 2024-06-18T13:05:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi167-74007-16"}]': finished 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi167-73991-3", "field": "max_bytes", "val": "4096"}]': finished 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]': finished 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi167-75207-1"}]': finished 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]': finished 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-4", "overlaypool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1683915713' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi167-75207-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: osdmap e87: 8 total, 8 up, 8 in 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3841079011' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-4", "overlaypool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4191097629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi167-73976-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:14.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi167-75207-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:14.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:14.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: 12.0 deep-scrub starts 2024-06-18T13:05:14.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: 12.0 deep-scrub ok 2024-06-18T13:05:14.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: pgmap v79: 796 pgs: 96 creating+peering, 280 unknown, 420 active+clean; 145 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 301 op/s 2024-06-18T13:05:14.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: 12.6 deep-scrub starts 2024-06-18T13:05:14.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: 12.6 deep-scrub ok 2024-06-18T13:05:14.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: 12.2 deep-scrub starts 2024-06-18T13:05:14.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: 12.2 deep-scrub ok 2024-06-18T13:05:14.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/774256593' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-18T13:05:14.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: threexx 2024-06-18T13:05:14.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:14.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-18T13:05:14.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3588247890' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi167-74007-16"}]': finished 2024-06-18T13:05:14.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi167-73991-3", "field": "max_bytes", "val": "4096"}]': finished 2024-06-18T13:05:14.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:14.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]': finished 2024-06-18T13:05:14.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi167-75207-1"}]': finished 2024-06-18T13:05:14.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]': finished 2024-06-18T13:05:14.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-4", "overlaypool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:14.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:14.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1683915713' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi167-75207-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:14.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: osdmap e87: 8 total, 8 up, 8 in 2024-06-18T13:05:14.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3841079011' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:14.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-4", "overlaypool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:14.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:14.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4191097629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi167-73976-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:14.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi167-75207-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:14.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:15.226 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:05:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:05:15.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: pgmap v82: 812 pgs: 128 creating+peering, 684 active+clean; 145 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 18 KiB/s rd, 16 KiB/s wr, 25 op/s 2024-06-18T13:05:15.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:15.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-06-18T13:05:15.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/774256593' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-06-18T13:05:15.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: fourxx 2024-06-18T13:05:15.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-06-18T13:05:15.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi167-74069-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi167-74069-10"}]': finished 2024-06-18T13:05:15.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-4", "overlaypool": "test-rados-api-smithi167-74266-4-cache"}]': finished 2024-06-18T13:05:15.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]': finished 2024-06-18T13:05:15.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4191097629' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi167-73976-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:15.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi167-75207-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:15.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:15.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1683915713' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi167-75207-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:15.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: osdmap e88: 8 total, 8 up, 8 in 2024-06-18T13:05:15.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-4-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:05:15.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:15.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:15.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/626807822' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:15.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/902417483' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:15.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:15.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-06-18T13:05:15.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:15.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:15.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi167-75207-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:15.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:15.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-4-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:05:15.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:15.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:15.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:15.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: pgmap v82: 812 pgs: 128 creating+peering, 684 active+clean; 145 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 18 KiB/s rd, 16 KiB/s wr, 25 op/s 2024-06-18T13:05:15.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:15.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-06-18T13:05:15.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/774256593' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-06-18T13:05:15.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: fourxx 2024-06-18T13:05:15.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-06-18T13:05:15.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi167-74069-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi167-74069-10"}]': finished 2024-06-18T13:05:15.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-4", "overlaypool": "test-rados-api-smithi167-74266-4-cache"}]': finished 2024-06-18T13:05:15.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]': finished 2024-06-18T13:05:15.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4191097629' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi167-73976-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:15.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi167-75207-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:15.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:15.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1683915713' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi167-75207-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:15.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: osdmap e88: 8 total, 8 up, 8 in 2024-06-18T13:05:15.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-4-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:05:15.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:15.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:15.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/626807822' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:15.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/902417483' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:15.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:15.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-06-18T13:05:15.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:15.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:15.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi167-75207-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:15.232 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:15.232 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-4-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:05:15.232 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:15.232 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:15.232 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:15.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: pgmap v82: 812 pgs: 128 creating+peering, 684 active+clean; 145 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 18 KiB/s rd, 16 KiB/s wr, 25 op/s 2024-06-18T13:05:15.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:15.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-06-18T13:05:15.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/774256593' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-06-18T13:05:15.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: fourxx 2024-06-18T13:05:15.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-06-18T13:05:15.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi167-74069-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi167-74069-10"}]': finished 2024-06-18T13:05:15.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-4", "overlaypool": "test-rados-api-smithi167-74266-4-cache"}]': finished 2024-06-18T13:05:15.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]': finished 2024-06-18T13:05:15.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4191097629' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi167-73976-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:15.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi167-75207-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:15.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:15.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1683915713' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi167-75207-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:15.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: osdmap e88: 8 total, 8 up, 8 in 2024-06-18T13:05:15.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-4-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:05:15.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:15.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:15.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/626807822' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:15.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/902417483' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:15.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:15.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-06-18T13:05:15.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:15.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:15.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi167-75207-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:15.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:15.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-4-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:05:15.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:15.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:15.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:15.923 INFO:tasks.workunit.client.0.smithi167.stdout: ec_io: Running main() from gmock_main.cc 2024-06-18T13:05:15.923 INFO:tasks.workunit.client.0.smithi167.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-06-18T13:05:15.923 INFO:tasks.workunit.client.0.smithi167.stdout: ec_io: [----------] Global test environment set-up. 2024-06-18T13:05:15.923 INFO:tasks.workunit.client.0.smithi167.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-06-18T13:05:15.923 INFO:tasks.workunit.client.0.smithi167.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-06-18T13:05:15.923 INFO:tasks.workunit.client.0.smithi167.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (13636 ms) 2024-06-18T13:05:15.923 INFO:tasks.workunit.client.0.smithi167.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-06-18T13:05:15.923 INFO:tasks.workunit.client.0.smithi167.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (7859 ms) 2024-06-18T13:05:15.923 INFO:tasks.workunit.client.0.smithi167.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (21495 ms total) 2024-06-18T13:05:15.923 INFO:tasks.workunit.client.0.smithi167.stdout: ec_io: 2024-06-18T13:05:15.923 INFO:tasks.workunit.client.0.smithi167.stdout: ec_io: [----------] Global test environment tear-down 2024-06-18T13:05:15.923 INFO:tasks.workunit.client.0.smithi167.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (21495 ms total) 2024-06-18T13:05:15.924 INFO:tasks.workunit.client.0.smithi167.stdout: ec_io: [ PASSED ] 2 tests. 2024-06-18T13:05:15.934 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-06-18T13:05:15.935 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-06-18T13:05:15.935 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-06-18T13:05:15.935 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-06-18T13:05:15.935 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: seed 74409 2024-06-18T13:05:15.935 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-06-18T13:05:15.935 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (258 ms) 2024-06-18T13:05:15.935 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-06-18T13:05:15.935 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (38 ms) 2024-06-18T13:05:15.935 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-06-18T13:05:15.935 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (3 ms) 2024-06-18T13:05:15.935 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-06-18T13:05:15.935 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (5 ms) 2024-06-18T13:05:15.935 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-06-18T13:05:15.936 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (24 ms) 2024-06-18T13:05:15.936 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (328 ms total) 2024-06-18T13:05:15.936 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: 2024-06-18T13:05:15.936 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-06-18T13:05:15.936 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-06-18T13:05:15.936 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1623 ms) 2024-06-18T13:05:15.936 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-06-18T13:05:15.936 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-06-18T13:05:15.936 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-06-18T13:05:15.936 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (4 ms) 2024-06-18T13:05:15.936 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-06-18T13:05:15.936 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (9 ms) 2024-06-18T13:05:15.936 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1636 ms total) 2024-06-18T13:05:15.937 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: 2024-06-18T13:05:15.937 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-06-18T13:05:15.937 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (21749 ms total) 2024-06-18T13:05:15.937 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-06-18T13:05:15.938 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: Running main() from gmock_main.cc 2024-06-18T13:05:15.938 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-06-18T13:05:15.938 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [----------] Global test environment set-up. 2024-06-18T13:05:15.938 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-06-18T13:05:15.938 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-06-18T13:05:15.938 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ OK ] LibRadosStat.Stat (323 ms) 2024-06-18T13:05:15.938 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-06-18T13:05:15.938 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (5 ms) 2024-06-18T13:05:15.938 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-06-18T13:05:15.938 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ OK ] LibRadosStat.StatNS (16 ms) 2024-06-18T13:05:15.938 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-06-18T13:05:15.938 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (2 ms) 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [----------] 5 tests from LibRadosStat (346 ms total) 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1094 ms) 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (12 ms) 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (168 ms) 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (19 ms) 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1293 ms total) 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: 2024-06-18T13:05:15.939 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [----------] Global test environment tear-down 2024-06-18T13:05:15.940 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (21756 ms total) 2024-06-18T13:05:15.940 INFO:tasks.workunit.client.0.smithi167.stdout: api_stat: [ PASSED ] 9 tests. 2024-06-18T13:05:15.947 INFO:tasks.workunit.client.0.smithi167.stdout:ibRadosIoECPP.OverlappingWriteRoundTripPP 2024-06-18T13:05:15.947 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (6 ms) 2024-06-18T13:05:15.947 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-06-18T13:05:15.947 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (3 ms) 2024-06-18T13:05:15.947 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-06-18T13:05:15.947 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (2 ms) 2024-06-18T13:05:15.947 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-06-18T13:05:15.947 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (3 ms) 2024-06-18T13:05:15.947 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-06-18T13:05:15.947 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (5 ms) 2024-06-18T13:05:15.947 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-06-18T13:05:15.947 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (2 ms) 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (7 ms) 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (3 ms) 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (9 ms) 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (5 ms) 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (7 ms) 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1169 ms total) 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [----------] Global test environment tear-down 2024-06-18T13:05:15.948 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (21899 ms total) 2024-06-18T13:05:15.949 INFO:tasks.workunit.client.0.smithi167.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-06-18T13:05:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:05:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-06-18T13:05:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]': finished 2024-06-18T13:05:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi167-74410-7"}]': finished 2024-06-18T13:05:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi167-75055-2"}]': finished 2024-06-18T13:05:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-4-cache", "mode": "writeback"}]': finished 2024-06-18T13:05:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:16.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]': finished 2024-06-18T13:05:16.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/626807822' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:16.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:16.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:16.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-4"}]: dispatch 2024-06-18T13:05:16.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: osdmap e89: 8 total, 8 up, 8 in 2024-06-18T13:05:16.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:16.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:16.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:16.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:16.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:16.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:16.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app2"}]: dispatch 2024-06-18T13:05:16.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:05:16.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-06-18T13:05:16.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:16.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]': finished 2024-06-18T13:05:16.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi167-74410-7"}]': finished 2024-06-18T13:05:16.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi167-75055-2"}]': finished 2024-06-18T13:05:16.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-4-cache", "mode": "writeback"}]': finished 2024-06-18T13:05:16.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:16.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]': finished 2024-06-18T13:05:16.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/626807822' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:16.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:16.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:16.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-4"}]: dispatch 2024-06-18T13:05:16.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: osdmap e89: 8 total, 8 up, 8 in 2024-06-18T13:05:16.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:16.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:16.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:16.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:16.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:16.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:16.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app2"}]: dispatch 2024-06-18T13:05:16.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:16.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-4"}]: dispatch 2024-06-18T13:05:16.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:16.232 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:16.232 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-4"}]: dispatch 2024-06-18T13:05:16.232 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:16.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:05:16.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-06-18T13:05:16.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:16.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi167-74409-7"}]': finished 2024-06-18T13:05:16.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi167-74410-7"}]': finished 2024-06-18T13:05:16.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi167-75055-2"}]': finished 2024-06-18T13:05:16.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-4-cache", "mode": "writeback"}]': finished 2024-06-18T13:05:16.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:16.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi167-74023-22"}]': finished 2024-06-18T13:05:16.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/626807822' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:16.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/829533142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:16.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3624363182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:16.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-4"}]: dispatch 2024-06-18T13:05:16.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: osdmap e89: 8 total, 8 up, 8 in 2024-06-18T13:05:16.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]: dispatch 2024-06-18T13:05:16.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi167-75055-2"}]: dispatch 2024-06-18T13:05:16.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi167-74410-7"}]: dispatch 2024-06-18T13:05:16.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:16.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]: dispatch 2024-06-18T13:05:16.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:16.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app2"}]: dispatch 2024-06-18T13:05:16.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:16.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-4"}]: dispatch 2024-06-18T13:05:16.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:16.904 INFO:tasks.workunit.client.0.smithi167.stdout:t: 2024-06-18T13:05:12.853513+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]': finished 2024-06-18T13:05:16.904 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:12.853594+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi167-75207-1"}]': finished 2024-06-18T13:05:16.904 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:12.853682+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi167-74507-1"}]': finished 2024-06-18T13:05:16.904 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:12.859128+0000 mon.b [INF] from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-4", "overlaypool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:16.904 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:12.862569+0000 mon.c [INF] from='client.? 172.21.15.167:0/3058253131' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:16.905 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:12.914045+0000 mon.b [INF] from='client.? 172.21.15.167:0/1683915713' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi167-75207-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:16.905 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:13.134637+0000 mon.b [INF] from='client.? 172.21.15.167:0/3841079011' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:16.905 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:13.138259+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-4", "overlaypool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:16.905 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:13.144613+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi167-74507-1"}]: dispatch 2024-06-18T13:05:16.905 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:13.145979+0000 mon.a [INF] from='client.? 172.21.15.167:0/4191097629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi167-73976-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:16.905 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:13.155783+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi167-75207-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:16.905 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: got: 2024-06-18T13:05:13.162674+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:16.905 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (11633 ms) 2024-06-18T13:05:16.905 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (22684 ms total) 2024-06-18T13:05:16.905 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: 2024-06-18T13:05:16.905 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [----------] Global test environment tear-down 2024-06-18T13:05:16.905 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (22685 ms total) 2024-06-18T13:05:16.905 INFO:tasks.workunit.client.0.smithi167.stdout: api_cmd: [ PASSED ] 4 tests. 2024-06-18T13:05:17.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-06-18T13:05:17.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-06-18T13:05:17.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-06-18T13:05:17.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-06-18T13:05:17.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: seed 74123 2024-06-18T13:05:17.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-06-18T13:05:17.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (93 ms) 2024-06-18T13:05:17.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-06-18T13:05:17.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (16 ms) 2024-06-18T13:05:17.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-06-18T13:05:17.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1168 ms) 2024-06-18T13:05:17.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-06-18T13:05:17.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1044 ms) 2024-06-18T13:05:17.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-06-18T13:05:17.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (42 ms) 2024-06-18T13:05:17.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-06-18T13:05:17.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (5 ms) 2024-06-18T13:05:17.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-06-18T13:05:17.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (28 ms) 2024-06-18T13:05:17.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-06-18T13:05:17.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (15 ms) 2024-06-18T13:05:17.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2411 ms total) 2024-06-18T13:05:17.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: 2024-06-18T13:05:17.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-06-18T13:05:17.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-06-18T13:05:17.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1618 ms) 2024-06-18T13:05:17.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-06-18T13:05:17.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (7 ms) 2024-06-18T13:05:17.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-06-18T13:05:17.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1005 ms) 2024-06-18T13:05:17.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-06-18T13:05:17.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1005 ms) 2024-06-18T13:05:17.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-06-18T13:05:17.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (4 ms) 2024-06-18T13:05:17.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-06-18T13:05:17.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (3 ms) 2024-06-18T13:05:17.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-06-18T13:05:17.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (4 ms) 2024-06-18T13:05:17.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-06-18T13:05:17.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (2 ms) 2024-06-18T13:05:17.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3648 ms total) 2024-06-18T13:05:17.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: 2024-06-18T13:05:17.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-06-18T13:05:17.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (23182 ms total) 2024-06-18T13:05:17.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-06-18T13:05:17.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: pgmap v85: 812 pgs: 232 unknown, 32 creating+peering, 548 active+clean; 145 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-06-18T13:05:17.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi167-75207-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi167-75207-2"}]': finished 2024-06-18T13:05:17.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]': finished 2024-06-18T13:05:17.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi167-75055-2"}]': finished 2024-06-18T13:05:17.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi167-74410-7"}]': finished 2024-06-18T13:05:17.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:17.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]': finished 2024-06-18T13:05:17.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10"}]': finished 2024-06-18T13:05:17.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:17.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-4"}]': finished 2024-06-18T13:05:17.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: osdmap e90: 8 total, 8 up, 8 in 2024-06-18T13:05:17.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:17.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:17.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2453388344' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi167-74332-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1711183959' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi167-74332-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/555935481' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi167-73976-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:17.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-06-18T13:05:17.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-18T13:05:17.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi167-73976-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74507-6", "pg_num": 4}]: dispatch 2024-06-18T13:05:17.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: pgmap v85: 812 pgs: 232 unknown, 32 creating+peering, 548 active+clean; 145 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-06-18T13:05:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi167-75207-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi167-75207-2"}]': finished 2024-06-18T13:05:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]': finished 2024-06-18T13:05:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi167-75055-2"}]': finished 2024-06-18T13:05:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi167-74410-7"}]': finished 2024-06-18T13:05:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]': finished 2024-06-18T13:05:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10"}]': finished 2024-06-18T13:05:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-4"}]': finished 2024-06-18T13:05:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: osdmap e90: 8 total, 8 up, 8 in 2024-06-18T13:05:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:17.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2453388344' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi167-74332-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1711183959' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi167-74332-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/555935481' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi167-73976-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: pgmap v85: 812 pgs: 232 unknown, 32 creating+peering, 548 active+clean; 145 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-06-18T13:05:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi167-75207-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi167-75207-2"}]': finished 2024-06-18T13:05:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi167-74023-22"}]': finished 2024-06-18T13:05:17.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi167-75055-2"}]': finished 2024-06-18T13:05:17.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2646123001' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi167-74410-7"}]': finished 2024-06-18T13:05:17.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:17.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi167-74409-7"}]': finished 2024-06-18T13:05:17.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi167-74123-10"}]': finished 2024-06-18T13:05:17.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi167-74124-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:17.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-4"}]': finished 2024-06-18T13:05:17.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: osdmap e90: 8 total, 8 up, 8 in 2024-06-18T13:05:17.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1442851202' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:17.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi167-74123-10"}]: dispatch 2024-06-18T13:05:17.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2453388344' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi167-74332-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1711183959' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi167-74332-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/555935481' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi167-73976-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]: dispatch 2024-06-18T13:05:17.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-06-18T13:05:17.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-18T13:05:17.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi167-73976-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74507-6", "pg_num": 4}]: dispatch 2024-06-18T13:05:17.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:17.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-06-18T13:05:17.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-18T13:05:17.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi167-73976-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:17.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74507-6", "pg_num": 4}]: dispatch 2024-06-18T13:05:17.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:18.008 INFO:tasks.workunit.client.0.smithi167.stdout: list: Running main() from gmock_main.cc 2024-06-18T13:05:18.008 INFO:tasks.workunit.client.0.smithi167.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-06-18T13:05:18.008 INFO:tasks.workunit.client.0.smithi167.stdout: list: [----------] Global test environment set-up. 2024-06-18T13:05:18.008 INFO:tasks.workunit.client.0.smithi167.stdout: list: [----------] 3 tests from NeoradosList 2024-06-18T13:05:18.008 INFO:tasks.workunit.client.0.smithi167.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-06-18T13:05:18.008 INFO:tasks.workunit.client.0.smithi167.stdout: list: [ OK ] NeoradosList.ListObjects (13883 ms) 2024-06-18T13:05:18.009 INFO:tasks.workunit.client.0.smithi167.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-06-18T13:05:18.009 INFO:tasks.workunit.client.0.smithi167.stdout: list: [ OK ] NeoradosList.ListObjectsNS (4806 ms) 2024-06-18T13:05:18.009 INFO:tasks.workunit.client.0.smithi167.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-06-18T13:05:18.009 INFO:tasks.workunit.client.0.smithi167.stdout: list: [ OK ] NeoradosList.ListObjectsMany (4864 ms) 2024-06-18T13:05:18.009 INFO:tasks.workunit.client.0.smithi167.stdout: list: [----------] 3 tests from NeoradosList (23553 ms total) 2024-06-18T13:05:18.009 INFO:tasks.workunit.client.0.smithi167.stdout: list: 2024-06-18T13:05:18.009 INFO:tasks.workunit.client.0.smithi167.stdout: list: [----------] Global test environment tear-down 2024-06-18T13:05:18.009 INFO:tasks.workunit.client.0.smithi167.stdout: list: [==========] 3 tests from 1 test suite ran. (23553 ms total) 2024-06-18T13:05:18.009 INFO:tasks.workunit.client.0.smithi167.stdout: list: [ PASSED ] 3 tests. 2024-06-18T13:05:18.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:05:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: pool 'PoolQuotaPP_smithi167-73991-3' is full (reached quota's max_bytes: 4 KiB) 2024-06-18T13:05:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-18T13:05:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:05:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi167-74123-10"}]': finished 2024-06-18T13:05:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi167-74332-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]': finished 2024-06-18T13:05:18.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:18.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-06-18T13:05:18.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi167-73976-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:18.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74507-6", "pg_num": 4}]': finished 2024-06-18T13:05:18.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: osdmap e91: 8 total, 8 up, 8 in 2024-06-18T13:05:18.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "tierpool": "test-rados-api-smithi167-74507-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:18.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-18T13:05:18.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:18.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "tierpool": "test-rados-api-smithi167-74507-6", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:18.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-06-18T13:05:18.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: osdmap e92: 8 total, 8 up, 8 in 2024-06-18T13:05:18.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "overlaypool": "test-rados-api-smithi167-74507-6"}]: dispatch 2024-06-18T13:05:18.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key1"}]: dispatch 2024-06-18T13:05:18.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1527134419' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:18.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1683915713' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:18.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:18.639 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: pool 'PoolQuotaPP_smithi167-73991-3' is full (reached quota's max_bytes: 4 KiB) 2024-06-18T13:05:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-18T13:05:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:05:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi167-74123-10"}]': finished 2024-06-18T13:05:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi167-74332-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]': finished 2024-06-18T13:05:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-06-18T13:05:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi167-73976-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74507-6", "pg_num": 4}]': finished 2024-06-18T13:05:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: osdmap e91: 8 total, 8 up, 8 in 2024-06-18T13:05:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "tierpool": "test-rados-api-smithi167-74507-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-18T13:05:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "tierpool": "test-rados-api-smithi167-74507-6", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-06-18T13:05:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: osdmap e92: 8 total, 8 up, 8 in 2024-06-18T13:05:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "overlaypool": "test-rados-api-smithi167-74507-6"}]: dispatch 2024-06-18T13:05:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key1"}]: dispatch 2024-06-18T13:05:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: pool 'PoolQuotaPP_smithi167-73991-3' is full (reached quota's max_bytes: 4 KiB) 2024-06-18T13:05:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-18T13:05:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:05:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1308990359' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi167-74123-10"}]': finished 2024-06-18T13:05:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi167-74332-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-4", "tierpool": "test-rados-api-smithi167-74266-4-cache"}]': finished 2024-06-18T13:05:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-06-18T13:05:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi167-73976-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74507-6", "pg_num": 4}]': finished 2024-06-18T13:05:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: osdmap e91: 8 total, 8 up, 8 in 2024-06-18T13:05:18.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "tierpool": "test-rados-api-smithi167-74507-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:18.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-18T13:05:18.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:18.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "tierpool": "test-rados-api-smithi167-74507-6", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:18.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-06-18T13:05:18.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: osdmap e92: 8 total, 8 up, 8 in 2024-06-18T13:05:18.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "overlaypool": "test-rados-api-smithi167-74507-6"}]: dispatch 2024-06-18T13:05:18.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key1"}]: dispatch 2024-06-18T13:05:18.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1527134419' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:18.731 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1683915713' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:18.731 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:18.731 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:18.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1527134419' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:18.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1683915713' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:18.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:18.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:19.009 INFO:tasks.workunit.client.0.smithi167.stdout: pool: Running main() from gmock_main.cc 2024-06-18T13:05:19.010 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-06-18T13:05:19.010 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [----------] Global test environment set-up. 2024-06-18T13:05:19.010 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-06-18T13:05:19.010 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-06-18T13:05:19.010 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [ OK ] NeoRadosPools.PoolList (9277 ms) 2024-06-18T13:05:19.010 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-06-18T13:05:19.010 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (2559 ms) 2024-06-18T13:05:19.010 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-06-18T13:05:19.010 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (2014 ms) 2024-06-18T13:05:19.010 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-06-18T13:05:19.011 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (5570 ms) 2024-06-18T13:05:19.011 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-06-18T13:05:19.011 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (2037 ms) 2024-06-18T13:05:19.011 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-06-18T13:05:19.011 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (3067 ms) 2024-06-18T13:05:19.011 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [----------] 6 tests from NeoRadosPools (24524 ms total) 2024-06-18T13:05:19.011 INFO:tasks.workunit.client.0.smithi167.stdout: pool: 2024-06-18T13:05:19.011 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [----------] Global test environment tear-down 2024-06-18T13:05:19.011 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [==========] 6 tests from 1 test suite ran. (24524 ms total) 2024-06-18T13:05:19.011 INFO:tasks.workunit.client.0.smithi167.stdout: pool: [ PASSED ] 6 tests. 2024-06-18T13:05:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: pgmap v88: 856 pgs: 308 unknown, 548 active+clean; 145 MiB data, 743 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:05:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "overlaypool": "test-rados-api-smithi167-74507-6"}]': finished 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key1"}]': finished 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi167-75207-2"}]': finished 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi167-74459-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1683915713' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: osdmap e93: 8 total, 8 up, 8 in 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74507-6", "mode": "writeback"}]: dispatch 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3702704430' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi167-73976-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1412687884' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi167-74459-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:20.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:20.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: pgmap v88: 856 pgs: 308 unknown, 548 active+clean; 145 MiB data, 743 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:05:20.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:20.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "overlaypool": "test-rados-api-smithi167-74507-6"}]': finished 2024-06-18T13:05:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key1"}]': finished 2024-06-18T13:05:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi167-75207-2"}]': finished 2024-06-18T13:05:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi167-74459-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1683915713' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: osdmap e93: 8 total, 8 up, 8 in 2024-06-18T13:05:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74507-6", "mode": "writeback"}]: dispatch 2024-06-18T13:05:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:20.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3702704430' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi167-73976-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:20.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1412687884' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:20.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi167-74459-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:20.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:20.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:20.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: pgmap v88: 856 pgs: 308 unknown, 548 active+clean; 145 MiB data, 743 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:05:20.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:20.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:20.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:20.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "overlaypool": "test-rados-api-smithi167-74507-6"}]': finished 2024-06-18T13:05:20.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2934899346' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi167-74124-1","app":"app1","key":"key1"}]': finished 2024-06-18T13:05:20.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:20.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi167-75207-2"}]': finished 2024-06-18T13:05:20.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:20.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi167-74459-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1683915713' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:20.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: osdmap e93: 8 total, 8 up, 8 in 2024-06-18T13:05:20.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74507-6", "mode": "writeback"}]: dispatch 2024-06-18T13:05:20.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi167-75207-2"}]: dispatch 2024-06-18T13:05:20.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3702704430' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi167-73976-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:20.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1412687884' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:20.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi167-74459-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:20.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:20.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:20.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:21.134 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: Running main() from gmock_main.cc 2024-06-18T13:05:21.134 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-06-18T13:05:21.134 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [----------] Global test environment set-up. 2024-06-18T13:05:21.134 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-06-18T13:05:21.134 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-06-18T13:05:21.134 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (325 ms) 2024-06-18T13:05:21.134 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-06-18T13:05:21.134 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLock.LockShared (9 ms) 2024-06-18T13:05:21.134 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-06-18T13:05:21.135 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1066 ms) 2024-06-18T13:05:21.135 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-06-18T13:05:21.135 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1039 ms) 2024-06-18T13:05:21.135 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-06-18T13:05:21.135 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (3 ms) 2024-06-18T13:05:21.135 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-06-18T13:05:21.135 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLock.Unlock (4 ms) 2024-06-18T13:05:21.135 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-06-18T13:05:21.135 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (3 ms) 2024-06-18T13:05:21.135 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-06-18T13:05:21.135 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-06-18T13:05:21.135 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [----------] 8 tests from LibRadosLock (2452 ms total) 2024-06-18T13:05:21.135 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: 2024-06-18T13:05:21.135 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (2110 ms) 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (17 ms) 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1018 ms) 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1051 ms) 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (14 ms) 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (11 ms) 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (4 ms) 2024-06-18T13:05:21.136 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-06-18T13:05:21.137 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (2 ms) 2024-06-18T13:05:21.137 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (4227 ms total) 2024-06-18T13:05:21.137 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: 2024-06-18T13:05:21.137 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [----------] Global test environment tear-down 2024-06-18T13:05:21.137 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (27033 ms total) 2024-06-18T13:05:21.137 INFO:tasks.workunit.client.0.smithi167.stdout: api_lock: [ PASSED ] 16 tests. 2024-06-18T13:05:21.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:21 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:21.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:21 smithi179 ceph-mon[33312]: pgmap v91: 616 pgs: 32 creating+peering, 64 unknown, 520 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:05:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:21 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:05:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74507-6", "mode": "writeback"}]': finished 2024-06-18T13:05:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi167-75207-2"}]': finished 2024-06-18T13:05:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3702704430' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi167-73976-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]': finished 2024-06-18T13:05:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:21 smithi179 ceph-mon[33312]: osdmap e94: 8 total, 8 up, 8 in 2024-06-18T13:05:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi167-75207-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:21.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:21.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[29496]: pgmap v91: 616 pgs: 32 creating+peering, 64 unknown, 520 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:05:21.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:21.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:05:21.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74507-6", "mode": "writeback"}]': finished 2024-06-18T13:05:21.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi167-75207-2"}]': finished 2024-06-18T13:05:21.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3702704430' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi167-73976-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:21.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:21.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]': finished 2024-06-18T13:05:21.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:21.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:21.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[29496]: osdmap e94: 8 total, 8 up, 8 in 2024-06-18T13:05:21.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:21.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi167-75207-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:21.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:21.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[37701]: pgmap v91: 616 pgs: 32 creating+peering, 64 unknown, 520 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:05:21.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:21.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:05:21.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74507-6", "mode": "writeback"}]': finished 2024-06-18T13:05:21.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi167-75207-2"}]': finished 2024-06-18T13:05:21.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3702704430' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi167-73976-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:21.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:21.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi167-74069-10"}]': finished 2024-06-18T13:05:21.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3010445150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:21.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:21.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[37701]: osdmap e94: 8 total, 8 up, 8 in 2024-06-18T13:05:21.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]: dispatch 2024-06-18T13:05:21.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi167-75207-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi167-74459-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]': finished 2024-06-18T13:05:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]': finished 2024-06-18T13:05:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi167-75207-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:22 smithi167 ceph-mon[37701]: osdmap e95: 8 total, 8 up, 8 in 2024-06-18T13:05:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi167-75207-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi167-75207-3"}]: dispatch 2024-06-18T13:05:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi167-74459-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]': finished 2024-06-18T13:05:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]': finished 2024-06-18T13:05:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi167-75207-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:22 smithi167 ceph-mon[29496]: osdmap e95: 8 total, 8 up, 8 in 2024-06-18T13:05:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi167-75207-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi167-75207-3"}]: dispatch 2024-06-18T13:05:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:23.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi167-74459-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]': finished 2024-06-18T13:05:23.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi167-74069-10"}]': finished 2024-06-18T13:05:23.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi167-75207-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:23.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:22 smithi179 ceph-mon[33312]: osdmap e95: 8 total, 8 up, 8 in 2024-06-18T13:05:23.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi167-75207-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi167-75207-3"}]: dispatch 2024-06-18T13:05:23.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:23.857 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-06-18T13:05:23.858 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-06-18T13:05:23.858 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-06-18T13:05:23.858 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-06-18T13:05:23.858 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-06-18T13:05:23.858 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: notify 2024-06-18T13:05:23.858 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (3158 ms) 2024-06-18T13:05:23.858 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-06-18T13:05:23.858 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (4 ms) 2024-06-18T13:05:23.858 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (3162 ms total) 2024-06-18T13:05:23.858 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: 2024-06-18T13:05:23.858 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-06-18T13:05:23.859 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-06-18T13:05:23.859 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: notify 2024-06-18T13:05:23.859 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (92 ms) 2024-06-18T13:05:23.859 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-06-18T13:05:23.859 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: notify 2024-06-18T13:05:23.859 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (4756 ms) 2024-06-18T13:05:23.859 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-06-18T13:05:23.859 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (4 ms) 2024-06-18T13:05:23.859 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-06-18T13:05:23.859 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (5 ms) 2024-06-18T13:05:23.859 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-06-18T13:05:23.859 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: handle_notify cookie 94585130089376 notify_id 403726925826 notifier_gid 15285 2024-06-18T13:05:23.859 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (8 ms) 2024-06-18T13:05:23.859 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-06-18T13:05:23.860 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: handle_notify cookie 94585130093616 notify_id 403726925825 notifier_gid 15285 2024-06-18T13:05:23.860 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (9 ms) 2024-06-18T13:05:23.860 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-06-18T13:05:23.860 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: handle_notify cookie 94585130093616 notify_id 403726925827 notifier_gid 15285 2024-06-18T13:05:23.860 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (11 ms) 2024-06-18T13:05:23.860 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-06-18T13:05:23.860 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: handle_notify cookie 94585130154672 notify_id 403726925828 notifier_gid 15285 2024-06-18T13:05:23.860 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (7 ms) 2024-06-18T13:05:23.860 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-06-18T13:05:23.860 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: handle_notify cookie 94585130154672 notify_id 403726925829 notifier_gid 15285 2024-06-18T13:05:23.860 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (13 ms) 2024-06-18T13:05:23.860 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-06-18T13:05:23.860 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: handle_notify cookie 94585130154672 notify_id 403726925830 notifier_gid 15285 2024-06-18T13:05:23.860 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (7 ms) 2024-06-18T13:05:23.861 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-06-18T13:05:23.861 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: trying... 2024-06-18T13:05:23.861 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: handle_notify cookie 94585130154672 notify_id 403726925826 notifier_gid 15285 2024-06-18T13:05:23.861 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: timed out 2024-06-18T13:05:23.861 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: flushing 2024-06-18T13:05:24.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: pgmap v94: 624 pgs: 136 unknown, 488 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-06-18T13:05:24.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: osdmap e96: 8 total, 8 up, 8 in 2024-06-18T13:05:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1182754707' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1102675319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-18T13:05:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi167-75207-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi167-75207-3"}]': finished 2024-06-18T13:05:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: osdmap e97: 8 total, 8 up, 8 in 2024-06-18T13:05:24.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:24.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:24.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:24.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:24 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:05:24.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: pgmap v94: 624 pgs: 136 unknown, 488 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-06-18T13:05:24.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:24.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:24.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: osdmap e96: 8 total, 8 up, 8 in 2024-06-18T13:05:24.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1182754707' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1102675319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-18T13:05:24.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi167-75207-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi167-75207-3"}]': finished 2024-06-18T13:05:24.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:24.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:24.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:24.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: osdmap e97: 8 total, 8 up, 8 in 2024-06-18T13:05:24.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:24.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:24.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:24.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:05:24.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: pgmap v94: 624 pgs: 136 unknown, 488 active+clean; 145 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-06-18T13:05:24.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:24.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:24.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: osdmap e96: 8 total, 8 up, 8 in 2024-06-18T13:05:24.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1182754707' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1102675319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:24.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-18T13:05:24.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi167-75207-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi167-75207-3"}]': finished 2024-06-18T13:05:24.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:24.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:24.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:24.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: osdmap e97: 8 total, 8 up, 8 in 2024-06-18T13:05:24.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:24.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:24.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:24.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:24 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:05:25.272 INFO:tasks.workunit.client.0.smithi167.stdout: api_wat api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-06-18T13:05:25.272 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: [----------] Global test environment set-up. 2024-06-18T13:05:25.272 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-06-18T13:05:25.272 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-06-18T13:05:25.272 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5062 ms) 2024-06-18T13:05:25.272 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-06-18T13:05:25.272 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (24 ms) 2024-06-18T13:05:25.272 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-06-18T13:05:25.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20018 ms) 2024-06-18T13:05:25.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-06-18T13:05:25.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: attempt 0 of 20 2024-06-18T13:05:25.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: failed to deregister: 2024-06-18T13:05:25.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: { 2024-06-18T13:05:25.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "epoch": 21, 2024-06-18T13:05:25.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "modified": "2024-06-18T13:05:19.464698+0000", 2024-06-18T13:05:25.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "services": { 2024-06-18T13:05:25.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "laundry": { 2024-06-18T13:05:25.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "daemons": { 2024-06-18T13:05:25.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "summary": "", 2024-06-18T13:05:25.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "close-test-pid74643": { 2024-06-18T13:05:25.273 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "start_epoch": 21, 2024-06-18T13:05:25.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "start_stamp": "2024-06-18T13:05:19.413626+0000", 2024-06-18T13:05:25.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "gid": 25354, 2024-06-18T13:05:25.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "addr": "172.21.15.167:0/2717190548", 2024-06-18T13:05:25.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "metadata": { 2024-06-18T13:05:25.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "arch": "x86_64", 2024-06-18T13:05:25.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_release": "squid", 2024-06-18T13:05:25.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-3124-g3781e3a5 (3781e3a592387d16e8020f5db99cdee4198f6d00) squid (rc)", 2024-06-18T13:05:25.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_version_short": "19.0.0-3124-g3781e3a5", 2024-06-18T13:05:25.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-06-18T13:05:25.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro": "centos", 2024-06-18T13:05:25.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-06-18T13:05:25.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro_version": "9", 2024-06-18T13:05:25.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "foo": "bar", 2024-06-18T13:05:25.274 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "hostname": "smithi167", 2024-06-18T13:05:25.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Jun 6 12:49:42 UTC 2024", 2024-06-18T13:05:25.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "kernel_version": "5.14.0-460.el9.x86_64", 2024-06-18T13:05:25.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "mem_swap_kb": "0", 2024-06-18T13:05:25.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "mem_total_kb": "32488116", 2024-06-18T13:05:25.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "os": "Linux", 2024-06-18T13:05:25.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "this": "that" 2024-06-18T13:05:25.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: }, 2024-06-18T13:05:25.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "task_status": {} 2024-06-18T13:05:25.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: }, 2024-06-18T13:05:25.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "pid74609": { 2024-06-18T13:05:25.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "start_epoch": 20, 2024-06-18T13:05:25.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "start_stamp": "2024-06-18T13:05:05.604702+0000", 2024-06-18T13:05:25.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "gid": 25118, 2024-06-18T13:05:25.275 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "addr": "172.21.15.167:0/1550686522", 2024-06-18T13:05:25.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "metadata": { 2024-06-18T13:05:25.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "arch": "x86_64", 2024-06-18T13:05:25.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_release": "squid", 2024-06-18T13:05:25.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-3124-g3781e3a5 (3781e3a592387d16e8020f5db99cdee4198f6d00) squid (rc)", 2024-06-18T13:05:25.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_version_short": "19.0.0-3124-g3781e3a5", 2024-06-18T13:05:25.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-06-18T13:05:25.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro": "centos", 2024-06-18T13:05:25.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-06-18T13:05:25.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro_version": "9", 2024-06-18T13:05:25.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "foo": "bar", 2024-06-18T13:05:25.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "hostname": "smithi167", 2024-06-18T13:05:25.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Jun 6 12:49:42 UTC 2024", 2024-06-18T13:05:25.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "kernel_version": "5.14.0-460.el9.x86_64", 2024-06-18T13:05:25.276 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "mem_swap_kb": "0", 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "mem_total_kb": "32488116", 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "os": "Linux", 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "this": "that" 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: }, 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "task_status": {} 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: }, 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "rgw": { 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "daemons": { 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "summary": "", 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "24350": { 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "start_epoch": 12, 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "start_stamp": "2024-06-18T13:02:37.536755+0000", 2024-06-18T13:05:25.277 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "gid": 24350, 2024-06-18T13:05:25.278 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "addr": "172.21.15.167:0/806756546", 2024-06-18T13:05:25.278 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "metadata": { 2024-06-18T13:05:25.278 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "arch": "x86_64", 2024-06-18T13:05:25.278 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_release": "squid", 2024-06-18T13:05:25.278 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-3124-g3781e3a5 (3781e3a592387d16e8020f5db99cdee4198f6d00) squid (rc)", 2024-06-18T13:05:25.278 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_version_short": "19.0.0-3124-g3781e3a5", 2024-06-18T13:05:25.278 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "container_hostname": "smithi167", 2024-06-18T13:05:25.278 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314", 2024-06-18T13:05:25.278 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-06-18T13:05:25.278 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro": "centos", 2024-06-18T13:05:25.278 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-06-18T13:05:25.278 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro_version": "9", 2024-06-18T13:05:25.278 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-06-18T13:05:25.278 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "frontend_type#0": "beast", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "hostname": "smithi167", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "id": "foo.a", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Jun 6 12:49:42 UTC 2024", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "kernel_version": "5.14.0-460.el9.x86_64", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "mem_swap_kb": "0", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "mem_total_kb": "32488116", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "num_handles": "1", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "os": "Linux", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "pid": "2", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "realm_id": "", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "realm_name": "", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "zone_id": "0e3b9be2-179c-49ea-8730-99cda783a4d1", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "zone_name": "default", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "zonegroup_id": "6138e376-0a52-44ff-a17c-f4a2d1126c05", 2024-06-18T13:05:25.279 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "zonegroup_name": "default" 2024-06-18T13:05:25.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:05:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:05:25.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[37701]: pgmap v97: 656 pgs: 50 creating+peering, 52 creating+activating, 54 unknown, 500 active+clean; 145 MiB data, 749 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 B/s wr, 3 op/s 2024-06-18T13:05:25.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:25.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4064027870' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi167-73976-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:25.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]': finished 2024-06-18T13:05:25.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4064027870' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi167-73976-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:25.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[37701]: osdmap e98: 8 total, 8 up, 8 in 2024-06-18T13:05:25.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:25.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:25.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[29496]: pgmap v97: 656 pgs: 50 creating+peering, 52 creating+activating, 54 unknown, 500 active+clean; 145 MiB data, 749 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 B/s wr, 3 op/s 2024-06-18T13:05:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4064027870' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi167-73976-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]': finished 2024-06-18T13:05:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4064027870' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi167-73976-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[29496]: osdmap e98: 8 total, 8 up, 8 in 2024-06-18T13:05:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:25.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:25.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:25.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:25.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:25 smithi179 ceph-mon[33312]: pgmap v97: 656 pgs: 50 creating+peering, 52 creating+activating, 54 unknown, 500 active+clean; 145 MiB data, 749 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 B/s wr, 3 op/s 2024-06-18T13:05:25.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4064027870' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi167-73976-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi167-74459-12"}]': finished 2024-06-18T13:05:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4064027870' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi167-73976-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:25 smithi179 ceph-mon[33312]: osdmap e98: 8 total, 8 up, 8 in 2024-06-18T13:05:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/807032567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]: dispatch 2024-06-18T13:05:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:25.660 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:25.660 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: data: 2024-06-18T13:05:25.660 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: pools: 42 pools, 1284 pgs 2024-06-18T13:05:25.661 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: objects: 253 objects, 587 KiB 2024-06-18T13:05:25.661 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: usage: 249 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:05:25.661 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: pgs: 7.944% pgs unknown 2024-06-18T13:05:25.661 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 64.330% pgs not active 2024-06-18T13:05:25.661 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 826 creating+peering 2024-06-18T13:05:25.661 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 356 active+clean 2024-06-18T13:05:25.661 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 102 unknown 2024-06-18T13:05:25.661 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:25.661 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: io: 2024-06-18T13:05:25.661 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: client: 8.2 KiB/s rd, 8.4 KiB/s wr, 71 op/s rd, 74 op/s wr 2024-06-18T13:05:25.661 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:25.661 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:25.661 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: [ OK ] LibRadosService.StatusFormat (5408 ms) 2024-06-18T13:05:25.661 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: [ RUN ] LibRadosService.Status 2024-06-18T13:05:25.662 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: [ OK ] LibRadosService.Status (20067 ms) 2024-06-18T13:05:25.662 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: [----------] 4 tests from LibRadosService (31371 ms total) 2024-06-18T13:05:25.662 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: 2024-06-18T13:05:25.662 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: [----------] Global test environment tear-down 2024-06-18T13:05:25.662 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: [==========] 4 tests from 1 test suite ran. (31371 ms total) 2024-06-18T13:05:25.662 INFO:tasks.workunit.client.0.smithi167.stdout: api_service: [ PASSED ] 4 tests. 2024-06-18T13:05:25.723 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-06-18T13:05:25.723 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-06-18T13:05:25.723 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-06-18T13:05:25.723 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: watch_notify_test_cb 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (2347 ms) 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94127913952336 err -107 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (1006 ms) 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94127913954656 err -107 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1041 ms) 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: watch_notify2_test_cb from 24836 notify_id 352187318272 cookie 94127913971472 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (11 ms) 2024-06-18T13:05:25.724 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-06-18T13:05:25.725 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: watch_notify2_test_cb from 24836 notify_id 352187318272 cookie 94127914023728 2024-06-18T13:05:25.725 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (5 ms) 2024-06-18T13:05:25.725 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-06-18T13:05:25.725 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: watch_notify2_test_cb from 24836 notify_id 352187318273 cookie 94127914023696 2024-06-18T13:05:25.725 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (5 ms) 2024-06-18T13:05:25.725 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-06-18T13:05:25.725 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: watch_notify2_test_cb from 24836 notify_id 352187318272 cookie 94127914003312 2024-06-18T13:05:25.725 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: watch_notify2_test_cb from 24836 notify_id 352187318272 cookie 94127914023696 2024-06-18T13:05:25.725 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (6 ms) 2024-06-18T13:05:25.725 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-06-18T13:05:25.725 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: watch_notify2_test_cb from 24836 notify_id 352187318273 cookie 94127914003312 2024-06-18T13:05:25.725 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: watch_notify2_test_cb from 24836 notify_id 356482285570 cookie 94127914003312 2024-06-18T13:05:25.726 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3005 ms) 2024-06-18T13:05:25.726 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-06-18T13:05:25.726 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-06-18T13:05:25.726 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94127914026208 err -107 2024-06-18T13:05:25.726 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: watch_notify2_test_cb from 24836 notify_id 386547056644 cookie 94127914026208 2024-06-18T13:05:25.726 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5007 ms) 2024-06-18T13:05:25.726 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-06-18T13:05:25.726 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-06-18T13:05:25.726 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94127914026208 err -107 2024-06-18T13:05:25.726 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1004 ms) 2024-06-18T13:05:25.726 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (13437 ms total) 2024-06-18T13:05:25.726 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: 2024-06-18T13:05:25.726 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-06-18T13:05:25.726 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-06-18T13:05:25.727 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: watch_notify_test_cb 2024-06-18T13:05:25.727 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1257 ms) 2024-06-18T13:05:25.727 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1257 ms total) 2024-06-18T13:05:25.727 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: 2024-06-18T13:05:25.727 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-06-18T13:05:25.727 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (31518 ms total) 2024-06-18T13:05:25.727 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-06-18T13:05:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2166163042' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]': finished 2024-06-18T13:05:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1687574354' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[29496]: osdmap e99: 8 total, 8 up, 8 in 2024-06-18T13:05:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2166163042' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]': finished 2024-06-18T13:05:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1687574354' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[37701]: osdmap e99: 8 total, 8 up, 8 in 2024-06-18T13:05:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:26.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2166163042' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:26 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi167-74459-12"}]': finished 2024-06-18T13:05:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1687574354' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:26 smithi179 ceph-mon[33312]: osdmap e99: 8 total, 8 up, 8 in 2024-06-18T13:05:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: pgmap v99: 624 pgs: 65 creating+peering, 20 creating+activating, 103 unknown, 436 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-18T13:05:28.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:28.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:28.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:28.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:28.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: osdmap e100: 8 total, 8 up, 8 in 2024-06-18T13:05:28.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1619300161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi167-73976-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-7"}]: dispatch 2024-06-18T13:05:28.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2395280931' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-7"}]: dispatch 2024-06-18T13:05:28.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3743410536' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi167-74374-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi167-74374-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:28.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1619300161' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi167-73976-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:28.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: pgmap v99: 624 pgs: 65 creating+peering, 20 creating+activating, 103 unknown, 436 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-18T13:05:28.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:28.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:28.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:28.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:28.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: osdmap e100: 8 total, 8 up, 8 in 2024-06-18T13:05:28.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1619300161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi167-73976-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-7"}]: dispatch 2024-06-18T13:05:28.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2395280931' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-7"}]: dispatch 2024-06-18T13:05:28.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3743410536' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi167-74374-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi167-74374-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:28.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1619300161' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi167-73976-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:28.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:05:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: pgmap v99: 624 pgs: 65 creating+peering, 20 creating+activating, 103 unknown, 436 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-18T13:05:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: osdmap e100: 8 total, 8 up, 8 in 2024-06-18T13:05:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1619300161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi167-73976-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-7"}]: dispatch 2024-06-18T13:05:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2395280931' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-7"}]: dispatch 2024-06-18T13:05:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3743410536' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi167-74374-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi167-74374-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1619300161' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi167-73976-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:29.653 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-7"}]': finished 2024-06-18T13:05:29.653 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:29.653 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi167-74374-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:29.653 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: pgmap v103: 656 pgs: 8 creating+peering, 20 creating+activating, 224 unknown, 404 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:05:29.653 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:29.653 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:29.653 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: osdmap e101: 8 total, 8 up, 8 in 2024-06-18T13:05:29.653 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:29.653 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: osdmap e102: 8 total, 8 up, 8 in 2024-06-18T13:05:29.653 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi167-75207-3"}]: dispatch 2024-06-18T13:05:29.654 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2791312632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:29.654 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:29.654 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:29.654 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:29.654 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:29 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:05:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-7"}]': finished 2024-06-18T13:05:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi167-74374-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: pgmap v103: 656 pgs: 8 creating+peering, 20 creating+activating, 224 unknown, 404 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:05:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: osdmap e101: 8 total, 8 up, 8 in 2024-06-18T13:05:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: osdmap e102: 8 total, 8 up, 8 in 2024-06-18T13:05:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi167-75207-3"}]: dispatch 2024-06-18T13:05:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2791312632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:05:29.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-7"}]': finished 2024-06-18T13:05:29.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:29.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi167-74374-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:29.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: pgmap v103: 656 pgs: 8 creating+peering, 20 creating+activating, 224 unknown, 404 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:05:29.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:29.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:29.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: osdmap e101: 8 total, 8 up, 8 in 2024-06-18T13:05:29.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:29.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: osdmap e102: 8 total, 8 up, 8 in 2024-06-18T13:05:29.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi167-75207-3"}]: dispatch 2024-06-18T13:05:29.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2791312632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:29.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:29.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:29.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:29.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:29 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:05:30.404 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: }, 2024-06-18T13:05:30.404 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "task_status": {} 2024-06-18T13:05:30.404 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:30.404 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:30.405 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:30.405 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:30.405 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:30.405 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: 2024-06-18T13:05:30.405 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: attempt 1 of 20 2024-06-18T13:05:30.405 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: failed to deregister: 2024-06-18T13:05:30.405 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: { 2024-06-18T13:05:30.405 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "epoch": 24, 2024-06-18T13:05:30.405 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "modified": "2024-06-18T13:05:27.469083+0000", 2024-06-18T13:05:30.405 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "services": { 2024-06-18T13:05:30.405 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "laundry": { 2024-06-18T13:05:30.405 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "daemons": { 2024-06-18T13:05:30.405 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "summary": "", 2024-06-18T13:05:30.406 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "close-test-pid74643": { 2024-06-18T13:05:30.406 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "start_epoch": 23, 2024-06-18T13:05:30.406 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "start_stamp": "2024-06-18T13:05:25.328376+0000", 2024-06-18T13:05:30.406 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "gid": 30538, 2024-06-18T13:05:30.406 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "addr": "172.21.15.167:0/1816584779", 2024-06-18T13:05:30.406 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "metadata": { 2024-06-18T13:05:30.406 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "arch": "x86_64", 2024-06-18T13:05:30.406 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_release": "squid", 2024-06-18T13:05:30.406 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-3124-g3781e3a5 (3781e3a592387d16e8020f5db99cdee4198f6d00) squid (rc)", 2024-06-18T13:05:30.406 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_version_short": "19.0.0-3124-g3781e3a5", 2024-06-18T13:05:30.406 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-06-18T13:05:30.406 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro": "centos", 2024-06-18T13:05:30.407 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-06-18T13:05:30.407 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro_version": "9", 2024-06-18T13:05:30.407 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "foo": "bar", 2024-06-18T13:05:30.407 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "hostname": "smithi167", 2024-06-18T13:05:30.407 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Jun 6 12:49:42 UTC 2024", 2024-06-18T13:05:30.407 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "kernel_version": "5.14.0-460.el9.x86_64", 2024-06-18T13:05:30.407 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "mem_swap_kb": "0", 2024-06-18T13:05:30.407 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "mem_total_kb": "32488116", 2024-06-18T13:05:30.407 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "os": "Linux", 2024-06-18T13:05:30.407 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "this": "that" 2024-06-18T13:05:30.407 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: }, 2024-06-18T13:05:30.407 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "task_status": {} 2024-06-18T13:05:30.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:30.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:30.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: }, 2024-06-18T13:05:30.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "rgw": { 2024-06-18T13:05:30.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "daemons": { 2024-06-18T13:05:30.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "summary": "", 2024-06-18T13:05:30.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "24350": { 2024-06-18T13:05:30.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "start_epoch": 12, 2024-06-18T13:05:30.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "start_stamp": "2024-06-18T13:02:37.536755+0000", 2024-06-18T13:05:30.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "gid": 24350, 2024-06-18T13:05:30.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "addr": "172.21.15.167:0/806756546", 2024-06-18T13:05:30.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "metadata": { 2024-06-18T13:05:30.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "arch": "x86_64", 2024-06-18T13:05:30.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_release": "squid", 2024-06-18T13:05:30.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-3124-g3781e3a5 (3781e3a592387d16e8020f5db99cdee4198f6d00) squid (rc)", 2024-06-18T13:05:30.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_version_short": "19.0.0-3124-g3781e3a5", 2024-06-18T13:05:30.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "container_hostname": "smithi167", 2024-06-18T13:05:30.913 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314 ec_list: Running main() from gmock_main.cc 2024-06-18T13:05:30.913 INFO:tasks.workunit.client.0.smithi167.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-06-18T13:05:30.913 INFO:tasks.workunit.client.0.smithi167.stdout: ec_list: [----------] Global test environment set-up. 2024-06-18T13:05:30.913 INFO:tasks.workunit.client.0.smithi167.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-06-18T13:05:30.913 INFO:tasks.workunit.client.0.smithi167.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-06-18T13:05:30.913 INFO:tasks.workunit.client.0.smithi167.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (18400 ms) 2024-06-18T13:05:30.913 INFO:tasks.workunit.client.0.smithi167.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-06-18T13:05:30.913 INFO:tasks.workunit.client.0.smithi167.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (7219 ms) 2024-06-18T13:05:30.913 INFO:tasks.workunit.client.0.smithi167.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-06-18T13:05:30.913 INFO:tasks.workunit.client.0.smithi167.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (10823 ms) 2024-06-18T13:05:30.913 INFO:tasks.workunit.client.0.smithi167.stdout: ec_list: [----------] 3 tests from NeoradosECList (36442 ms total) 2024-06-18T13:05:30.913 INFO:tasks.workunit.client.0.smithi167.stdout: ec_list: 2024-06-18T13:05:30.914 INFO:tasks.workunit.client.0.smithi167.stdout: ec_list: [----------] Global test environment tear-down 2024-06-18T13:05:30.914 INFO:tasks.workunit.client.0.smithi167.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (36442 ms total) 2024-06-18T13:05:30.914 INFO:tasks.workunit.client.0.smithi167.stdout: ec_list: [ PASSED ] 3 tests. 2024-06-18T13:05:31.212 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:05:31.212 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi167-75207-3"}]': finished 2024-06-18T13:05:31.213 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:31.213 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:30 smithi179 ceph-mon[33312]: osdmap e103: 8 total, 8 up, 8 in 2024-06-18T13:05:31.213 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7"}]: dispatch 2024-06-18T13:05:31.213 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi167-75207-3"}]: dispatch 2024-06-18T13:05:31.213 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7"}]: dispatch 2024-06-18T13:05:31.213 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:31.213 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1266896079' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:31.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:05:31.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi167-75207-3"}]': finished 2024-06-18T13:05:31.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:31.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[29496]: osdmap e103: 8 total, 8 up, 8 in 2024-06-18T13:05:31.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7"}]: dispatch 2024-06-18T13:05:31.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi167-75207-3"}]: dispatch 2024-06-18T13:05:31.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7"}]: dispatch 2024-06-18T13:05:31.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:31.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1266896079' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:31.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:05:31.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi167-75207-3"}]': finished 2024-06-18T13:05:31.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:31.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[37701]: osdmap e103: 8 total, 8 up, 8 in 2024-06-18T13:05:31.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7"}]: dispatch 2024-06-18T13:05:31.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi167-75207-3"}]: dispatch 2024-06-18T13:05:31.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7"}]: dispatch 2024-06-18T13:05:31.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:31.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1266896079' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: pgmap v106: 648 pgs: 10 creating+activating, 89 creating+peering, 29 unknown, 520 active+clean; 145 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-06-18T13:05:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi167-75207-3"}]': finished 2024-06-18T13:05:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7"}]': finished 2024-06-18T13:05:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: osdmap e104: 8 total, 8 up, 8 in 2024-06-18T13:05:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1191712502' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3371625087' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi167-73976-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2194885412' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi167-73976-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:32.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:32.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi167-73976-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:32.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:32.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:32 smithi179 ceph-mon[33312]: osdmap e105: 8 total, 8 up, 8 in 2024-06-18T13:05:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: pgmap v106: 648 pgs: 10 creating+activating, 89 creating+peering, 29 unknown, 520 active+clean; 145 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-06-18T13:05:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi167-75207-3"}]': finished 2024-06-18T13:05:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7"}]': finished 2024-06-18T13:05:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: osdmap e104: 8 total, 8 up, 8 in 2024-06-18T13:05:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1191712502' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3371625087' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi167-73976-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2194885412' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi167-73976-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:32.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:32.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi167-73976-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:32.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:32.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[37701]: osdmap e105: 8 total, 8 up, 8 in 2024-06-18T13:05:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: pgmap v106: 648 pgs: 10 creating+activating, 89 creating+peering, 29 unknown, 520 active+clean; 145 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-06-18T13:05:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3334800902' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi167-75207-3"}]': finished 2024-06-18T13:05:32.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-7"}]': finished 2024-06-18T13:05:32.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: osdmap e104: 8 total, 8 up, 8 in 2024-06-18T13:05:32.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1191712502' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3371625087' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi167-73976-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2194885412' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi167-73976-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:32.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:32.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74696-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:32.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi167-73976-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:32.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi167-73991-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:32.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:32 smithi167 ceph-mon[29496]: osdmap e105: 8 total, 8 up, 8 in 2024-06-18T13:05:34.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:33 smithi167 ceph-mon[29496]: pgmap v108: 712 pgs: 48 creating+peering, 144 unknown, 520 active+clean; 145 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-06-18T13:05:34.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:33 smithi167 ceph-mon[29496]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:34.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:33 smithi167 ceph-mon[29496]: osdmap e106: 8 total, 8 up, 8 in 2024-06-18T13:05:34.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:34.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:33 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:34.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:34.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4"}]: dispatch 2024-06-18T13:05:34.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:33 smithi167 ceph-mon[37701]: pgmap v108: 712 pgs: 48 creating+peering, 144 unknown, 520 active+clean; 145 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-06-18T13:05:34.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:33 smithi167 ceph-mon[37701]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:34.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:33 smithi167 ceph-mon[37701]: osdmap e106: 8 total, 8 up, 8 in 2024-06-18T13:05:34.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:34.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:33 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:34.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:34.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4"}]: dispatch 2024-06-18T13:05:34.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:33 smithi179 ceph-mon[33312]: pgmap v108: 712 pgs: 48 creating+peering, 144 unknown, 520 active+clean; 145 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-06-18T13:05:34.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:33 smithi179 ceph-mon[33312]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:34.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:33 smithi179 ceph-mon[33312]: osdmap e106: 8 total, 8 up, 8 in 2024-06-18T13:05:34.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:34.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:33 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:34.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:34.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4"}]: dispatch 2024-06-18T13:05:35.057 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: Running main() from gmock_main.cc 2024-06-18T13:05:35.057 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-06-18T13:05:35.057 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [----------] Global test environment set-up. 2024-06-18T13:05:35.057 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-06-18T13:05:35.057 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-06-18T13:05:35.057 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (10762 ms) 2024-06-18T13:05:35.057 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-06-18T13:05:35.058 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (7865 ms) 2024-06-18T13:05:35.058 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-06-18T13:05:35.058 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (2797 ms) 2024-06-18T13:05:35.058 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-06-18T13:05:35.058 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (3091 ms) 2024-06-18T13:05:35.058 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-06-18T13:05:35.058 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (4373 ms) 2024-06-18T13:05:35.058 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-06-18T13:05:35.058 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (3454 ms) 2024-06-18T13:05:35.058 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-06-18T13:05:35.058 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (8198 ms) 2024-06-18T13:05:35.058 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (40540 ms total) 2024-06-18T13:05:35.058 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: 2024-06-18T13:05:35.058 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [----------] Global test environment tear-down 2024-06-18T13:05:35.059 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (40540 ms total) 2024-06-18T13:05:35.059 INFO:tasks.workunit.client.0.smithi167.stdout: write_operations: [ PASSED ] 7 tests. 2024-06-18T13:05:35.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: pgmap v111: 616 pgs: 27 creating+peering, 65 unknown, 524 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 758 B/s rd, 1011 B/s wr, 2 op/s 2024-06-18T13:05:35.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:35.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:35.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:35.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4"}]': finished 2024-06-18T13:05:35.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: osdmap e107: 8 total, 8 up, 8 in 2024-06-18T13:05:35.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1514670663' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi167-73991-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3525005333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi167-73976-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2751977191' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "tierpool": "test-rados-api-smithi167-74507-6"}]: dispatch 2024-06-18T13:05:35.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:35.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi167-73991-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi167-73976-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:35.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:35.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:35.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: pgmap v111: 616 pgs: 27 creating+peering, 65 unknown, 524 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 758 B/s rd, 1011 B/s wr, 2 op/s 2024-06-18T13:05:35.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:35.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:35.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:35.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4"}]': finished 2024-06-18T13:05:35.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: osdmap e107: 8 total, 8 up, 8 in 2024-06-18T13:05:35.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1514670663' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi167-73991-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3525005333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi167-73976-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2751977191' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "tierpool": "test-rados-api-smithi167-74507-6"}]: dispatch 2024-06-18T13:05:35.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:35.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi167-73991-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi167-73976-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:35.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:35.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:35.480 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:05:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:05:35.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: pgmap v111: 616 pgs: 27 creating+peering, 65 unknown, 524 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 758 B/s rd, 1011 B/s wr, 2 op/s 2024-06-18T13:05:35.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:35.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:35.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:35.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4"}]': finished 2024-06-18T13:05:35.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: osdmap e107: 8 total, 8 up, 8 in 2024-06-18T13:05:35.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1514670663' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi167-73991-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3525005333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi167-73976-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2751977191' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "tierpool": "test-rados-api-smithi167-74507-6"}]: dispatch 2024-06-18T13:05:35.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:35.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi167-73991-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi167-73976-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:35.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:35.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:35.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:36.121 INFO:tasks.workunit.client.0.smithi167.stdout:", 2024-06-18T13:05:36.122 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-06-18T13:05:36.122 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro": "centos", 2024-06-18T13:05:36.122 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-06-18T13:05:36.122 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro_version": "9", 2024-06-18T13:05:36.122 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-06-18T13:05:36.122 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "frontend_type#0": "beast", 2024-06-18T13:05:36.122 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "hostname": "smithi167", 2024-06-18T13:05:36.122 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "id": "foo.a", 2024-06-18T13:05:36.122 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Jun 6 12:49:42 UTC 2024", 2024-06-18T13:05:36.122 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "kernel_version": "5.14.0-460.el9.x86_64", 2024-06-18T13:05:36.122 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "mem_swap_kb": "0", 2024-06-18T13:05:36.122 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "mem_total_kb": "32488116", 2024-06-18T13:05:36.123 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "num_handles": "1", 2024-06-18T13:05:36.123 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "os": "Linux", 2024-06-18T13:05:36.123 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "pid": "2", 2024-06-18T13:05:36.123 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "realm_id": "", 2024-06-18T13:05:36.123 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "realm_name": "", 2024-06-18T13:05:36.123 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "zone_id": "0e3b9be2-179c-49ea-8730-99cda783a4d1", 2024-06-18T13:05:36.123 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "zone_name": "default", 2024-06-18T13:05:36.123 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "zonegroup_id": "6138e376-0a52-44ff-a17c-f4a2d1126c05", 2024-06-18T13:05:36.123 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "zonegroup_name": "default" 2024-06-18T13:05:36.123 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: }, 2024-06-18T13:05:36.123 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "task_status": {} 2024-06-18T13:05:36.123 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:36.123 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:36.124 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:36.124 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:36.124 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:36.124 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: 2024-06-18T13:05:36.124 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: attempt 2 of 20 2024-06-18T13:05:36.124 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: failed to deregister: 2024-06-18T13:05:36.124 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: { 2024-06-18T13:05:36.124 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "epoch": 26, 2024-06-18T13:05:36.124 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "modified": "2024-06-18T13:05:31.470371+0000", 2024-06-18T13:05:36.124 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "services": { 2024-06-18T13:05:36.124 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "laundry": { 2024-06-18T13:05:36.124 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "daemons": { 2024-06-18T13:05:36.124 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "summary": "", 2024-06-18T13:05:36.125 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "close-test-pid74643": { 2024-06-18T13:05:36.125 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "start_epoch": 26, 2024-06-18T13:05:36.125 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "start_stamp": "2024-06-18T13:05:31.344771+0000", 2024-06-18T13:05:36.125 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "gid": 23328, 2024-06-18T13:05:36.125 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "addr": "172.21.15.167:0/1191122128", 2024-06-18T13:05:36.125 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "metadata": { 2024-06-18T13:05:36.125 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "arch": "x86_64", 2024-06-18T13:05:36.125 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_release": "squid", 2024-06-18T13:05:36.125 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-3124-g3781e3a5 (3781e3a592387d16e8020f5db99cdee4198f6d00) squid (rc)", 2024-06-18T13:05:36.125 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_version_short": "19.0.0-3124-g3781e3a5", 2024-06-18T13:05:36.125 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-06-18T13:05:36.125 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro": "centos", 2024-06-18T13:05:36.126 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-06-18T13:05:36.126 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro_version": "9", 2024-06-18T13:05:36.126 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "foo": "bar", 2024-06-18T13:05:36.126 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "hostname": "smithi167", 2024-06-18T13:05:36.126 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Jun 6 12:49:42 UTC 2024", 2024-06-18T13:05:36.126 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "kernel_version": "5.14.0-460.el9.x86_64", 2024-06-18T13:05:36.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:05:36.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "tierpool": "test-rados-api-smithi167-74507-6"}]': finished 2024-06-18T13:05:36.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi167-73991-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:36.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi167-73976-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:36.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:36.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:36.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: osdmap e108: 8 total, 8 up, 8 in 2024-06-18T13:05:36.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-9"}]: dispatch 2024-06-18T13:05:36.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-9"}]: dispatch 2024-06-18T13:05:36.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi167-74507-6", "pool2": "test-rados-api-smithi167-74507-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:05:36.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:36.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:36.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:36.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:36.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-9"}]': finished 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi167-74507-6", "pool2": "test-rados-api-smithi167-74507-6", "yes_i_really_really_mean_it": true}]': finished 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi167-74332-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: osdmap e109: 8 total, 8 up, 8 in 2024-06-18T13:05:36.390 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi167-74332-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.390 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-9", "mode": "writeback"}]: dispatch 2024-06-18T13:05:36.390 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-9", "mode": "writeback"}]: dispatch 2024-06-18T13:05:36.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:05:36.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "tierpool": "test-rados-api-smithi167-74507-6"}]': finished 2024-06-18T13:05:36.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi167-73991-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:36.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi167-73976-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:36.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:36.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:36.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: osdmap e108: 8 total, 8 up, 8 in 2024-06-18T13:05:36.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-9"}]: dispatch 2024-06-18T13:05:36.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-9"}]: dispatch 2024-06-18T13:05:36.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi167-74507-6", "pool2": "test-rados-api-smithi167-74507-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:05:36.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:36.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:36.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:36.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:36.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:36.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-06-18T13:05:36.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-06-18T13:05:36.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-06-18T13:05:36.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-06-18T13:05:36.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-06-18T13:05:36.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-06-18T13:05:36.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-06-18T13:05:36.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-06-18T13:05:36.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-06-18T13:05:36.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-06-18T13:05:36.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-9"}]': finished 2024-06-18T13:05:36.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi167-74507-6", "pool2": "test-rados-api-smithi167-74507-6", "yes_i_really_really_mean_it": true}]': finished 2024-06-18T13:05:36.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:36.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi167-74332-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: osdmap e109: 8 total, 8 up, 8 in 2024-06-18T13:05:36.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi167-74332-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-9", "mode": "writeback"}]: dispatch 2024-06-18T13:05:36.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-9", "mode": "writeback"}]: dispatch 2024-06-18T13:05:36.481 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:05:36.481 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi167-74507-4", "tierpool": "test-rados-api-smithi167-74507-6"}]': finished 2024-06-18T13:05:36.481 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi167-73991-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:36.481 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi167-73976-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:36.481 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:36.481 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:36.482 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: osdmap e108: 8 total, 8 up, 8 in 2024-06-18T13:05:36.482 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-9"}]: dispatch 2024-06-18T13:05:36.482 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-9"}]: dispatch 2024-06-18T13:05:36.482 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi167-74507-6", "pool2": "test-rados-api-smithi167-74507-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:05:36.482 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.482 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:36.482 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:36.482 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.482 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.482 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.482 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:36.483 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:36.483 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:36.483 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-06-18T13:05:36.483 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-06-18T13:05:36.483 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-06-18T13:05:36.483 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-06-18T13:05:36.483 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-06-18T13:05:36.483 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-06-18T13:05:36.483 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-06-18T13:05:36.483 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-06-18T13:05:36.483 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-06-18T13:05:36.483 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-06-18T13:05:36.484 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-9"}]': finished 2024-06-18T13:05:36.484 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4004427596' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi167-74507-6", "pool2": "test-rados-api-smithi167-74507-6", "yes_i_really_really_mean_it": true}]': finished 2024-06-18T13:05:36.484 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:36.484 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi167-74332-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.484 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: osdmap e109: 8 total, 8 up, 8 in 2024-06-18T13:05:36.484 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi167-74332-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:36.484 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-9", "mode": "writeback"}]: dispatch 2024-06-18T13:05:36.484 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-9", "mode": "writeback"}]: dispatch 2024-06-18T13:05:37.388 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "mem_ch_notify_pp: flushed 2024-06-18T13:05:37.388 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3006 ms) 2024-06-18T13:05:37.388 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-06-18T13:05:37.389 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: trying... 2024-06-18T13:05:37.389 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: handle_notify cookie 94585130154672 notify_id 416611827719 notifier_gid 15285 2024-06-18T13:05:37.389 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: timed out 2024-06-18T13:05:37.389 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: flushing 2024-06-18T13:05:37.389 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: flushed 2024-06-18T13:05:37.389 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3012 ms) 2024-06-18T13:05:37.389 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-06-18T13:05:37.389 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: List watches 2024-06-18T13:05:37.389 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: notify2 2024-06-18T13:05:37.390 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: handle_notify cookie 94585130154672 notify_id 429496729602 notifier_gid 15285 2024-06-18T13:05:37.390 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: notify2 done 2024-06-18T13:05:37.390 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: watch_check 2024-06-18T13:05:37.390 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: unwatch2 2024-06-18T13:05:37.390 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: flushing 2024-06-18T13:05:37.390 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: done 2024-06-18T13:05:37.390 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3167 ms) 2024-06-18T13:05:37.390 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-06-18T13:05:37.390 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: List watches 2024-06-18T13:05:37.391 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: notify2 2024-06-18T13:05:37.391 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: handle_notify cookie 94585130154672 notify_id 442381631491 notifier_gid 15285 2024-06-18T13:05:37.391 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: notify2 done 2024-06-18T13:05:37.391 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: watch_check 2024-06-18T13:05:37.391 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: unwatch2 2024-06-18T13:05:37.391 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: flushing 2024-06-18T13:05:37.391 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: done 2024-06-18T13:05:37.391 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3020 ms) 2024-06-18T13:05:37.391 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (17117 ms total) 2024-06-18T13:05:37.391 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: 2024-06-18T13:05:37.391 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-06-18T13:05:37.391 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (43163 ms total) 2024-06-18T13:05:37.392 INFO:tasks.workunit.client.0.smithi167.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-06-18T13:05:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-06-18T13:05:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-06-18T13:05:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-06-18T13:05:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-06-18T13:05:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-06-18T13:05:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-06-18T13:05:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-06-18T13:05:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: 2024-06-18T13:05:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-06-18T13:05:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-06-18T13:05:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (14054 ms) 2024-06-18T13:05:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-06-18T13:05:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (4800 ms) 2024-06-18T13:05:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-06-18T13:05:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (2793 ms) 2024-06-18T13:05:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-06-18T13:05:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (5193 ms) 2024-06-18T13:05:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-06-18T13:05:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3350 ms) 2024-06-18T13:05:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-06-18T13:05:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (5003 ms) 2024-06-18T13:05:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-06-18T13:05:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (7914 ms) 2024-06-18T13:05:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (43109 ms total) 2024-06-18T13:05:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: 2024-06-18T13:05:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-06-18T13:05:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (43109 ms total) 2024-06-18T13:05:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-06-18T13:05:37.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: pgmap v114: 680 pgs: 32 creating+peering, 192 unknown, 456 active+clean; 145 MiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:05:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-06-18T13:05:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-06-18T13:05:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-06-18T13:05:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-06-18T13:05:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-06-18T13:05:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-06-18T13:05:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-06-18T13:05:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-06-18T13:05:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-06-18T13:05:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-06-18T13:05:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: 12.9 deep-scrub starts 2024-06-18T13:05:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: 12.9 deep-scrub ok 2024-06-18T13:05:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: 12.0 deep-scrub starts 2024-06-18T13:05:37.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: 12.0 deep-scrub ok 2024-06-18T13:05:37.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1687892050' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:37.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:37.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: 12.8 deep-scrub starts 2024-06-18T13:05:37.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:37.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:05:37.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-9", "mode": "writeback"}]': finished 2024-06-18T13:05:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: pgmap v114: 680 pgs: 32 creating+peering, 192 unknown, 456 active+clean; 145 MiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:05:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-06-18T13:05:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-06-18T13:05:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-06-18T13:05:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-06-18T13:05:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-06-18T13:05:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-06-18T13:05:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-06-18T13:05:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-06-18T13:05:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-06-18T13:05:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-06-18T13:05:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: 12.9 deep-scrub starts 2024-06-18T13:05:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: 12.9 deep-scrub ok 2024-06-18T13:05:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: 12.0 deep-scrub starts 2024-06-18T13:05:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: 12.0 deep-scrub ok 2024-06-18T13:05:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1687892050' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: 12.8 deep-scrub starts 2024-06-18T13:05:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:05:37.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-9", "mode": "writeback"}]': finished 2024-06-18T13:05:37.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: pgmap v114: 680 pgs: 32 creating+peering, 192 unknown, 456 active+clean; 145 MiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:05:37.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-06-18T13:05:37.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-06-18T13:05:37.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-06-18T13:05:37.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-06-18T13:05:37.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-06-18T13:05:37.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-06-18T13:05:37.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-06-18T13:05:37.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-06-18T13:05:37.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-06-18T13:05:37.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-06-18T13:05:37.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: 12.9 deep-scrub starts 2024-06-18T13:05:37.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: 12.9 deep-scrub ok 2024-06-18T13:05:37.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: 12.0 deep-scrub starts 2024-06-18T13:05:37.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: 12.0 deep-scrub ok 2024-06-18T13:05:37.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1687892050' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:37.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:37.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: 12.8 deep-scrub starts 2024-06-18T13:05:37.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:37.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:05:37.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-9", "mode": "writeback"}]': finished 2024-06-18T13:05:38.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:05:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: 12.8 deep-scrub ok 2024-06-18T13:05:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: 12.1 deep-scrub starts 2024-06-18T13:05:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: 12.1 deep-scrub ok 2024-06-18T13:05:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: 12.3 deep-scrub starts 2024-06-18T13:05:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: 12.3 deep-scrub ok 2024-06-18T13:05:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: 12.2 deep-scrub starts 2024-06-18T13:05:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: 12.2 deep-scrub ok 2024-06-18T13:05:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: osdmap e110: 8 total, 8 up, 8 in 2024-06-18T13:05:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3461215429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi167-73991-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi167-74332-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]': finished 2024-06-18T13:05:38.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: 12.5 deep-scrub starts 2024-06-18T13:05:38.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: 12.5 deep-scrub ok 2024-06-18T13:05:38.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: osdmap e111: 8 total, 8 up, 8 in 2024-06-18T13:05:38.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:05:38.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4055198451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi167-73976-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:38.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:38.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi167-73991-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:38.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:05:38.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:38.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:38 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: 12.8 deep-scrub ok 2024-06-18T13:05:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: 12.1 deep-scrub starts 2024-06-18T13:05:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: 12.1 deep-scrub ok 2024-06-18T13:05:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: 12.3 deep-scrub starts 2024-06-18T13:05:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: 12.3 deep-scrub ok 2024-06-18T13:05:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: 12.2 deep-scrub starts 2024-06-18T13:05:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: 12.2 deep-scrub ok 2024-06-18T13:05:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: osdmap e110: 8 total, 8 up, 8 in 2024-06-18T13:05:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3461215429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi167-73991-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi167-74332-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]': finished 2024-06-18T13:05:38.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: 12.5 deep-scrub starts 2024-06-18T13:05:38.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: 12.5 deep-scrub ok 2024-06-18T13:05:38.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: osdmap e111: 8 total, 8 up, 8 in 2024-06-18T13:05:38.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:05:38.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4055198451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi167-73976-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:38.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:38.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi167-73991-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:38.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:05:38.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:38.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:38.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: 12.8 deep-scrub ok 2024-06-18T13:05:38.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: 12.1 deep-scrub starts 2024-06-18T13:05:38.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: 12.1 deep-scrub ok 2024-06-18T13:05:38.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: 12.3 deep-scrub starts 2024-06-18T13:05:38.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: 12.3 deep-scrub ok 2024-06-18T13:05:38.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: 12.2 deep-scrub starts 2024-06-18T13:05:38.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: 12.2 deep-scrub ok 2024-06-18T13:05:38.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:38.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: osdmap e110: 8 total, 8 up, 8 in 2024-06-18T13:05:38.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:38.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3461215429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi167-73991-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:38.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi167-74332-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]': finished 2024-06-18T13:05:38.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: 12.5 deep-scrub starts 2024-06-18T13:05:38.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: 12.5 deep-scrub ok 2024-06-18T13:05:38.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: osdmap e111: 8 total, 8 up, 8 in 2024-06-18T13:05:38.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:05:38.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4055198451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi167-73976-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:38.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:38.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi167-73991-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:38.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:05:38.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:38.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:38 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:39.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:39 smithi179 ceph-mon[33312]: pgmap v118: 556 pgs: 32 creating+peering, 136 unknown, 388 active+clean; 145 MiB data, 786 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:05:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:39 smithi179 ceph-mon[33312]: 12.4 deep-scrub starts 2024-06-18T13:05:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:39 smithi179 ceph-mon[33312]: 12.4 deep-scrub ok 2024-06-18T13:05:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:39 smithi179 ceph-mon[33312]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-18T13:05:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4055198451' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi167-73976-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:39 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:05:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:39 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi167-73991-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:39 smithi179 ceph-mon[33312]: osdmap e112: 8 total, 8 up, 8 in 2024-06-18T13:05:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9"}]: dispatch 2024-06-18T13:05:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:39 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9"}]: dispatch 2024-06-18T13:05:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:05:39.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[29496]: pgmap v118: 556 pgs: 32 creating+peering, 136 unknown, 388 active+clean; 145 MiB data, 786 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:05:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[29496]: 12.4 deep-scrub starts 2024-06-18T13:05:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[29496]: 12.4 deep-scrub ok 2024-06-18T13:05:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[29496]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-18T13:05:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4055198451' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi167-73976-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:05:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi167-73991-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[29496]: osdmap e112: 8 total, 8 up, 8 in 2024-06-18T13:05:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9"}]: dispatch 2024-06-18T13:05:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9"}]: dispatch 2024-06-18T13:05:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:05:39.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[37701]: pgmap v118: 556 pgs: 32 creating+peering, 136 unknown, 388 active+clean; 145 MiB data, 786 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:05:39.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[37701]: 12.4 deep-scrub starts 2024-06-18T13:05:39.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[37701]: 12.4 deep-scrub ok 2024-06-18T13:05:39.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[37701]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-18T13:05:39.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4055198451' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi167-73976-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:39.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:05:39.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi167-73991-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:39.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[37701]: osdmap e112: 8 total, 8 up, 8 in 2024-06-18T13:05:39.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9"}]: dispatch 2024-06-18T13:05:39.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9"}]: dispatch 2024-06-18T13:05:39.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:39.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:39.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:05:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:40 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:05:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9"}]': finished 2024-06-18T13:05:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:40 smithi179 ceph-mon[33312]: osdmap e113: 8 total, 8 up, 8 in 2024-06-18T13:05:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:40.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:40 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:05:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9"}]': finished 2024-06-18T13:05:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:40 smithi167 ceph-mon[29496]: osdmap e113: 8 total, 8 up, 8 in 2024-06-18T13:05:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:40 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:05:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-9"}]': finished 2024-06-18T13:05:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:40 smithi167 ceph-mon[37701]: osdmap e113: 8 total, 8 up, 8 in 2024-06-18T13:05:40.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:40.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:41.222 INFO:tasks.workunit.client.0.smithi167.stdout:swap_kb": "0", 2024-06-18T13:05:41.223 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "mem_total_kb": "32488116", 2024-06-18T13:05:41.223 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "os": "Linux", 2024-06-18T13:05:41.223 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "this": "that" 2024-06-18T13:05:41.223 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: }, 2024-06-18T13:05:41.223 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "task_status": {} 2024-06-18T13:05:41.223 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:41.223 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:41.223 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: }, 2024-06-18T13:05:41.223 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "rgw": { 2024-06-18T13:05:41.223 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "daemons": { 2024-06-18T13:05:41.223 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "summary": "", 2024-06-18T13:05:41.224 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "24350": { 2024-06-18T13:05:41.224 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "start_epoch": 12, 2024-06-18T13:05:41.224 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "start_stamp": "2024-06-18T13:02:37.536755+0000", 2024-06-18T13:05:41.224 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "gid": 24350, 2024-06-18T13:05:41.224 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "addr": "172.21.15.167:0/806756546", 2024-06-18T13:05:41.224 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "metadata": { 2024-06-18T13:05:41.224 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "arch": "x86_64", 2024-06-18T13:05:41.224 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_release": "squid", 2024-06-18T13:05:41.224 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-3124-g3781e3a5 (3781e3a592387d16e8020f5db99cdee4198f6d00) squid (rc)", 2024-06-18T13:05:41.224 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "ceph_version_short": "19.0.0-3124-g3781e3a5", 2024-06-18T13:05:41.224 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "container_hostname": "smithi167", 2024-06-18T13:05:41.225 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314", 2024-06-18T13:05:41.225 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-06-18T13:05:41.225 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro": "centos", 2024-06-18T13:05:41.225 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-06-18T13:05:41.225 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "distro_version": "9", 2024-06-18T13:05:41.225 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-06-18T13:05:41.225 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "frontend_type#0": "beast", 2024-06-18T13:05:41.225 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "hostname": "smithi167", 2024-06-18T13:05:41.225 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "id": "foo.a", 2024-06-18T13:05:41.225 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Jun 6 12:49:42 UTC 2024", 2024-06-18T13:05:41.225 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "kernel_version": "5.14.0-460.el9.x86_64", 2024-06-18T13:05:41.225 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "mem_swap_kb": "0", 2024-06-18T13:05:41.225 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "mem_total_kb": "32488116", 2024-06-18T13:05:41.226 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "num_handles": "1", 2024-06-18T13:05:41.226 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "os": "Linux", 2024-06-18T13:05:41.226 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "pid": "2", 2024-06-18T13:05:41.226 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "realm_id": "", 2024-06-18T13:05:41.226 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "realm_name": "", 2024-06-18T13:05:41.226 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "zone_id": "0e3b9be2-179c-49ea-8730-99cda783a4d1", 2024-06-18T13:05:41.226 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "zone_name": "default", 2024-06-18T13:05:41.226 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "zonegroup_id": "6138e376-0a52-44ff-a17c-f4a2d1126c05", 2024-06-18T13:05:41.226 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "zonegroup_name": "default" 2024-06-18T13:05:41.226 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: }, 2024-06-18T13:05:41.226 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: "task_status": {} 2024-06-18T13:05:41.226 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:41.227 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:41.227 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:41.227 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:41.227 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: } 2024-06-18T13:05:41.227 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: 2024-06-18T13:05:41.227 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: attempt 3 of 20 2024-06-18T13:05:41.227 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (21844 ms) 2024-06-18T13:05:41.227 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (46948 ms total) 2024-06-18T13:05:41.227 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: 2024-06-18T13:05:41.227 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: [----------] Global test environment tear-down 2024-06-18T13:05:41.228 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (46948 ms total) 2024-06-18T13:05:41.228 INFO:tasks.workunit.client.0.smithi167.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-06-18T13:05:41.548 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-06-18T13:05:41.548 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [----------] Global test environment set-up. 2024-06-18T13:05:41.548 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-06-18T13:05:41.548 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-06-18T13:05:41.548 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-06-18T13:05:41.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-06-18T13:05:41.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: 2024-06-18T13:05:41.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-06-18T13:05:41.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-06-18T13:05:41.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-06-18T13:05:41.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: 2024-06-18T13:04:54.169+0000 7fd5886f9780 -1 failed for service _ceph-mon._tcp 2024-06-18T13:05:41.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: 2024-06-18T13:04:54.169+0000 7fd5886f9780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-06-18T13:05:41.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (50 ms) 2024-06-18T13:05:41.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-06-18T13:05:41.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2017 ms) 2024-06-18T13:05:41.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2067 ms total) 2024-06-18T13:05:41.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: 2024-06-18T13:05:41.549 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-06-18T13:05:41.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-06-18T13:05:41.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: started 0x5584a4af5660 2024-06-18T13:05:41.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: started 0x5584a4e34ab0 2024-06-18T13:05:41.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: started 2 aios 2024-06-18T13:05:41.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: waiting 0x5584a4af5660 2024-06-18T13:05:41.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: waiting 0x5584a4e34ab0 2024-06-18T13:05:41.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: done. 2024-06-18T13:05:41.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (12913 ms) 2024-06-18T13:05:41.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (12913 ms total) 2024-06-18T13:05:41.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: 2024-06-18T13:05:41.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-06-18T13:05:41.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-06-18T13:05:41.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-06-18T13:05:41.550 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ OK ] LibRadosMisc.Exec (15 ms) 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (7 ms) 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (3 ms) 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5841 ms) 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (22199 ms) 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [----------] 8 tests from LibRadosMisc (28065 ms total) 2024-06-18T13:05:41.551 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: 2024-06-18T13:05:41.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [----------] Global test environment tear-down 2024-06-18T13:05:41.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (47415 ms total) 2024-06-18T13:05:41.552 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc: [ PASSED ] 12 tests. 2024-06-18T13:05:41.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:41 smithi179 ceph-mon[33312]: pgmap v121: 524 pgs: 32 creating+peering, 4 creating+activating, 64 unknown, 424 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 21 MiB/s wr, 5 op/s 2024-06-18T13:05:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:41 smithi179 ceph-mon[33312]: osdmap e114: 8 total, 8 up, 8 in 2024-06-18T13:05:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2907812293' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi167-73976-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1152608572' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi167-73991-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi167-73976-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi167-73991-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:41 smithi179 ceph-mon[33312]: 12.6 deep-scrub starts 2024-06-18T13:05:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:41 smithi179 ceph-mon[33312]: 12.6 deep-scrub ok 2024-06-18T13:05:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2502467807' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:41 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[37701]: pgmap v121: 524 pgs: 32 creating+peering, 4 creating+activating, 64 unknown, 424 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 21 MiB/s wr, 5 op/s 2024-06-18T13:05:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[37701]: osdmap e114: 8 total, 8 up, 8 in 2024-06-18T13:05:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2907812293' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi167-73976-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1152608572' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi167-73991-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:41.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi167-73976-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:41.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi167-73991-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:41.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:41.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[37701]: 12.6 deep-scrub starts 2024-06-18T13:05:41.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[37701]: 12.6 deep-scrub ok 2024-06-18T13:05:41.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2502467807' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:41.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:41.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[29496]: pgmap v121: 524 pgs: 32 creating+peering, 4 creating+activating, 64 unknown, 424 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 21 MiB/s wr, 5 op/s 2024-06-18T13:05:41.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74291-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:41.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[29496]: osdmap e114: 8 total, 8 up, 8 in 2024-06-18T13:05:41.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2907812293' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi167-73976-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:41.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1152608572' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi167-73991-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:41.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi167-73976-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:41.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi167-73991-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:41.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:41.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[29496]: 12.6 deep-scrub starts 2024-06-18T13:05:41.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[29496]: 12.6 deep-scrub ok 2024-06-18T13:05:41.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2502467807' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:41.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:41 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-18T13:05:42.388 INFO:tasks.workunit.client.0.smithi167.stdout: misc: Running main() from gmock_main.cc 2024-06-18T13:05:42.388 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-06-18T13:05:42.388 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [----------] Global test environment set-up. 2024-06-18T13:05:42.388 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-06-18T13:05:42.388 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-06-18T13:05:42.388 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ OK ] NeoRadosMisc.Version (9926 ms) 2024-06-18T13:05:42.388 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-06-18T13:05:42.388 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (1931 ms) 2024-06-18T13:05:42.388 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-06-18T13:05:42.388 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ OK ] NeoRadosMisc.LongName (5287 ms) 2024-06-18T13:05:42.388 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-06-18T13:05:42.388 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (4320 ms) 2024-06-18T13:05:42.388 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (3083 ms) 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (4388 ms) 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ OK ] NeoRadosMisc.Exec (3464 ms) 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (4099 ms) 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (3050 ms) 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ OK ] NeoRadosMisc.BigObject (3356 ms) 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (2062 ms) 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-06-18T13:05:42.389 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (2952 ms) 2024-06-18T13:05:42.390 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [----------] 12 tests from NeoRadosMisc (47918 ms total) 2024-06-18T13:05:42.390 INFO:tasks.workunit.client.0.smithi167.stdout: misc: 2024-06-18T13:05:42.390 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [----------] Global test environment tear-down 2024-06-18T13:05:42.390 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [==========] 12 tests from 1 test suite ran. (47918 ms total) 2024-06-18T13:05:42.390 INFO:tasks.workunit.client.0.smithi167.stdout: misc: [ PASSED ] 12 tests. 2024-06-18T13:05:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[29496]: pgmap v123: 588 pgs: 4 creating+activating, 160 unknown, 424 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-06-18T13:05:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi167-73976-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi167-73991-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[29496]: osdmap e115: 8 total, 8 up, 8 in 2024-06-18T13:05:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74291-10", "tierpool":"test-rados-api-smithi167-74291-10-cache", "force_nonempty":""}]: dispatch 2024-06-18T13:05:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[29496]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74291-10", "tierpool":"test-rados-api-smithi167-74291-10-cache", "force_nonempty":""}]': finished 2024-06-18T13:05:42.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[29496]: osdmap e116: 8 total, 8 up, 8 in 2024-06-18T13:05:42.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74291-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:42.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:42.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:42.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[37701]: pgmap v123: 588 pgs: 4 creating+activating, 160 unknown, 424 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-06-18T13:05:42.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi167-73976-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:42.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi167-73991-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:42.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[37701]: osdmap e115: 8 total, 8 up, 8 in 2024-06-18T13:05:42.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74291-10", "tierpool":"test-rados-api-smithi167-74291-10-cache", "force_nonempty":""}]: dispatch 2024-06-18T13:05:42.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:42.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[37701]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:42.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74291-10", "tierpool":"test-rados-api-smithi167-74291-10-cache", "force_nonempty":""}]': finished 2024-06-18T13:05:42.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[37701]: osdmap e116: 8 total, 8 up, 8 in 2024-06-18T13:05:42.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74291-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:42.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:42.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:43.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:42 smithi179 ceph-mon[33312]: pgmap v123: 588 pgs: 4 creating+activating, 160 unknown, 424 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-06-18T13:05:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi167-73976-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi167-73991-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:42 smithi179 ceph-mon[33312]: osdmap e115: 8 total, 8 up, 8 in 2024-06-18T13:05:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74291-10", "tierpool":"test-rados-api-smithi167-74291-10-cache", "force_nonempty":""}]: dispatch 2024-06-18T13:05:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:42 smithi179 ceph-mon[33312]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74291-10", "tierpool":"test-rados-api-smithi167-74291-10-cache", "force_nonempty":""}]': finished 2024-06-18T13:05:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:42 smithi179 ceph-mon[33312]: osdmap e116: 8 total, 8 up, 8 in 2024-06-18T13:05:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74291-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[37701]: 12.7 deep-scrub starts 2024-06-18T13:05:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[37701]: 12.7 deep-scrub ok 2024-06-18T13:05:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74291-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[37701]: osdmap e117: 8 total, 8 up, 8 in 2024-06-18T13:05:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1645288211' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2912725561' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi167-73976-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74291-10", "tierpool":"test-rados-api-smithi167-74291-10-cache"}]: dispatch 2024-06-18T13:05:43.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[29496]: 12.7 deep-scrub starts 2024-06-18T13:05:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[29496]: 12.7 deep-scrub ok 2024-06-18T13:05:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74291-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[29496]: osdmap e117: 8 total, 8 up, 8 in 2024-06-18T13:05:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1645288211' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2912725561' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi167-73976-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:43.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74291-10", "tierpool":"test-rados-api-smithi167-74291-10-cache"}]: dispatch 2024-06-18T13:05:43.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:44.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:43 smithi179 ceph-mon[33312]: 12.7 deep-scrub starts 2024-06-18T13:05:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:43 smithi179 ceph-mon[33312]: 12.7 deep-scrub ok 2024-06-18T13:05:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74291-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:43 smithi179 ceph-mon[33312]: osdmap e117: 8 total, 8 up, 8 in 2024-06-18T13:05:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1645288211' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2912725561' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi167-73976-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74291-10", "tierpool":"test-rados-api-smithi167-74291-10-cache"}]: dispatch 2024-06-18T13:05:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:45.397 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:05:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:05:45.696 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[37701]: pgmap v127: 556 pgs: 4 creating+activating, 192 unknown, 360 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:05:45.696 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2912725561' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi167-73976-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:45.696 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:45.696 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74291-10", "tierpool":"test-rados-api-smithi167-74291-10-cache"}]': finished 2024-06-18T13:05:45.696 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[37701]: osdmap e118: 8 total, 8 up, 8 in 2024-06-18T13:05:45.696 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:45.696 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:45.696 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:45.697 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[29496]: pgmap v127: 556 pgs: 4 creating+activating, 192 unknown, 360 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:05:45.697 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2912725561' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi167-73976-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:45.697 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:45.697 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74291-10", "tierpool":"test-rados-api-smithi167-74291-10-cache"}]': finished 2024-06-18T13:05:45.697 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[29496]: osdmap e118: 8 total, 8 up, 8 in 2024-06-18T13:05:45.697 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:45.697 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:45.697 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:45 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:45.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:45 smithi179 ceph-mon[33312]: pgmap v127: 556 pgs: 4 creating+activating, 192 unknown, 360 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:05:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2912725561' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi167-73976-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:45 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4095924437' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74291-10", "tierpool":"test-rados-api-smithi167-74291-10-cache"}]': finished 2024-06-18T13:05:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:45 smithi179 ceph-mon[33312]: osdmap e118: 8 total, 8 up, 8 in 2024-06-18T13:05:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:45 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:46.401 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: Running main() from gmock_main.cc 2024-06-18T13:05:46.401 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-06-18T13:05:46.401 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [----------] Global test environment set-up. 2024-06-18T13:05:46.401 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-06-18T13:05:46.401 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-06-18T13:05:46.401 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ OK ] LibRadosPools.PoolList (11130 ms) 2024-06-18T13:05:46.402 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-06-18T13:05:46.402 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (6573 ms) 2024-06-18T13:05:46.402 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-06-18T13:05:46.402 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (3328 ms) 2024-06-18T13:05:46.402 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-06-18T13:05:46.402 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (2836 ms) 2024-06-18T13:05:46.402 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-06-18T13:05:46.402 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3100 ms) 2024-06-18T13:05:46.402 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-06-18T13:05:46.403 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5736 ms) 2024-06-18T13:05:46.403 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-06-18T13:05:46.403 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5563 ms) 2024-06-18T13:05:46.403 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-06-18T13:05:46.403 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (6048 ms) 2024-06-18T13:05:46.403 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-06-18T13:05:46.403 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7945 ms) 2024-06-18T13:05:46.403 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [----------] 9 tests from LibRadosPools (52259 ms total) 2024-06-18T13:05:46.403 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: 2024-06-18T13:05:46.403 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [----------] Global test environment tear-down 2024-06-18T13:05:46.403 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (52259 ms total) 2024-06-18T13:05:46.403 INFO:tasks.workunit.client.0.smithi167.stdout: api_pool: [ PASSED ] 9 tests. 2024-06-18T13:05:46.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:46.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-11"}]: dispatch 2024-06-18T13:05:46.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[37701]: osdmap e119: 8 total, 8 up, 8 in 2024-06-18T13:05:46.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-11"}]: dispatch 2024-06-18T13:05:46.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[37701]: pgmap v130: 492 pgs: 64 unknown, 428 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-06-18T13:05:46.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:46.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-11"}]': finished 2024-06-18T13:05:46.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[37701]: osdmap e120: 8 total, 8 up, 8 in 2024-06-18T13:05:46.666 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:46.666 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-11"}]: dispatch 2024-06-18T13:05:46.667 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[29496]: osdmap e119: 8 total, 8 up, 8 in 2024-06-18T13:05:46.667 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-11"}]: dispatch 2024-06-18T13:05:46.667 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[29496]: pgmap v130: 492 pgs: 64 unknown, 428 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-06-18T13:05:46.667 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:46.667 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-11"}]': finished 2024-06-18T13:05:46.667 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:46 smithi167 ceph-mon[29496]: osdmap e120: 8 total, 8 up, 8 in 2024-06-18T13:05:46.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-11"}]: dispatch 2024-06-18T13:05:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:46 smithi179 ceph-mon[33312]: osdmap e119: 8 total, 8 up, 8 in 2024-06-18T13:05:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-11"}]: dispatch 2024-06-18T13:05:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:46 smithi179 ceph-mon[33312]: pgmap v130: 492 pgs: 64 unknown, 428 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-06-18T13:05:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-11"}]': finished 2024-06-18T13:05:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:46 smithi179 ceph-mon[33312]: osdmap e120: 8 total, 8 up, 8 in 2024-06-18T13:05:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-11", "mode": "writeback"}]: dispatch 2024-06-18T13:05:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3449419709' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi167-73976-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-11", "mode": "writeback"}]: dispatch 2024-06-18T13:05:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi167-73976-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3733724260' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi167-73991-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi167-73991-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:05:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-11", "mode": "writeback"}]': finished 2024-06-18T13:05:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi167-73976-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi167-73991-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: osdmap e121: 8 total, 8 up, 8 in 2024-06-18T13:05:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:47.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-11", "mode": "writeback"}]: dispatch 2024-06-18T13:05:47.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3449419709' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi167-73976-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:47.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-11", "mode": "writeback"}]: dispatch 2024-06-18T13:05:47.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi167-73976-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:47.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3733724260' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi167-73991-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:47.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi167-73991-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:47.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:47.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:47.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:05:47.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-11", "mode": "writeback"}]': finished 2024-06-18T13:05:47.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi167-73976-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:47.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi167-73991-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:47.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: osdmap e121: 8 total, 8 up, 8 in 2024-06-18T13:05:47.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:47.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:47.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-11", "mode": "writeback"}]: dispatch 2024-06-18T13:05:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3449419709' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi167-73976-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-11", "mode": "writeback"}]: dispatch 2024-06-18T13:05:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi167-73976-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3733724260' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi167-73991-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi167-73991-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:05:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-11", "mode": "writeback"}]': finished 2024-06-18T13:05:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi167-73976-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi167-73991-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:47.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: osdmap e121: 8 total, 8 up, 8 in 2024-06-18T13:05:47.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:47.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:48.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:48 smithi167 ceph-mon[37701]: pgmap v133: 516 pgs: 128 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-06-18T13:05:48.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]': finished 2024-06-18T13:05:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:48 smithi167 ceph-mon[37701]: osdmap e122: 8 total, 8 up, 8 in 2024-06-18T13:05:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:48 smithi167 ceph-mon[29496]: pgmap v133: 516 pgs: 128 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-06-18T13:05:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]': finished 2024-06-18T13:05:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:48 smithi167 ceph-mon[29496]: osdmap e122: 8 total, 8 up, 8 in 2024-06-18T13:05:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:48.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:05:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:05:48.893 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:48 smithi179 ceph-mon[33312]: pgmap v133: 516 pgs: 128 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-06-18T13:05:48.893 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:48.893 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi167-74332-10"}]': finished 2024-06-18T13:05:48.893 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:48 smithi179 ceph-mon[33312]: osdmap e122: 8 total, 8 up, 8 in 2024-06-18T13:05:48.893 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1038592707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:48.893 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]: dispatch 2024-06-18T13:05:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]': finished 2024-06-18T13:05:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:05:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[37701]: osdmap e123: 8 total, 8 up, 8 in 2024-06-18T13:05:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11"}]: dispatch 2024-06-18T13:05:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/812107760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi167-73976-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:49.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11"}]: dispatch 2024-06-18T13:05:49.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi167-73976-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:49.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3295314097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi167-73991-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:49.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:49.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:49.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:49.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:49.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:49.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]': finished 2024-06-18T13:05:49.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:05:49.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[29496]: osdmap e123: 8 total, 8 up, 8 in 2024-06-18T13:05:49.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11"}]: dispatch 2024-06-18T13:05:49.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/812107760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi167-73976-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:49.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11"}]: dispatch 2024-06-18T13:05:49.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi167-73976-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:49.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3295314097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi167-73991-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:49.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:49 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:05:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:05:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi167-74332-10"}]': finished 2024-06-18T13:05:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:05:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:49 smithi179 ceph-mon[33312]: osdmap e123: 8 total, 8 up, 8 in 2024-06-18T13:05:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11"}]: dispatch 2024-06-18T13:05:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/812107760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi167-73976-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11"}]: dispatch 2024-06-18T13:05:50.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi167-73976-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:50.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3295314097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi167-73991-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:50.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:50.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: pgmap v136: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 259 KiB/s wr, 9 op/s 2024-06-18T13:05:50.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:50.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:50.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:50.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:50.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:50.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:50.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:05:50.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11"}]': finished 2024-06-18T13:05:50.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi167-73976-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:50.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3295314097' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi167-73991-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:50.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:50.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: osdmap e124: 8 total, 8 up, 8 in 2024-06-18T13:05:50.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:50.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:50.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:50.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: pgmap v136: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 259 KiB/s wr, 9 op/s 2024-06-18T13:05:50.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:50.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:50.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:50.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:50.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:50.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:50.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:05:50.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11"}]': finished 2024-06-18T13:05:50.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi167-73976-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:50.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3295314097' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi167-73991-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:50.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:50.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: osdmap e124: 8 total, 8 up, 8 in 2024-06-18T13:05:50.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:50.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:50.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:51.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: pgmap v136: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 259 KiB/s wr, 9 op/s 2024-06-18T13:05:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:05:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:05:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-11"}]': finished 2024-06-18T13:05:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi167-73976-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3295314097' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi167-73991-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:51.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:05:51.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: osdmap e124: 8 total, 8 up, 8 in 2024-06-18T13:05:51.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:51.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:05:51.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:52.395 INFO:tasks.workunit.client.0.smithi167.stdout: io: Running main() from gmock_main.cc 2024-06-18T13:05:52.395 INFO:tasks.workunit.client.0.smithi167.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-06-18T13:05:52.395 INFO:tasks.workunit.client.0.smithi167.stdout: io: [----------] Global test environment set-up. 2024-06-18T13:05:52.396 INFO:tasks.workunit.client.0.smithi167.stdout: io: [----------] 14 tests from NeoRadosIo 2024-06-18T13:05:52.396 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-06-18T13:05:52.396 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ OK ] NeoRadosIo.Limits (13919 ms) 2024-06-18T13:05:52.396 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-06-18T13:05:52.396 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (4801 ms) 2024-06-18T13:05:52.396 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-06-18T13:05:52.396 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ OK ] NeoRadosIo.ReadOp (2800 ms) 2024-06-18T13:05:52.396 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-06-18T13:05:52.396 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ OK ] NeoRadosIo.SparseRead (3078 ms) 2024-06-18T13:05:52.396 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-06-18T13:05:52.396 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ OK ] NeoRadosIo.RoundTrip (3621 ms) 2024-06-18T13:05:52.396 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-06-18T13:05:52.396 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (4218 ms) 2024-06-18T13:05:52.396 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-06-18T13:05:52.397 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (4070 ms) 2024-06-18T13:05:52.397 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-06-18T13:05:52.397 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (3080 ms) 2024-06-18T13:05:52.397 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-06-18T13:05:52.397 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (3366 ms) 2024-06-18T13:05:52.397 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-06-18T13:05:52.397 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ OK ] NeoRadosIo.Trunc (3185 ms) 2024-06-18T13:05:52.397 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-06-18T13:05:52.397 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ OK ] NeoRadosIo.Remove (2836 ms) 2024-06-18T13:05:52.397 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-06-18T13:05:52.397 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (3010 ms) 2024-06-18T13:05:52.397 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-06-18T13:05:52.397 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ OK ] NeoRadosIo.RmXattr (2992 ms) 2024-06-18T13:05:52.398 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-06-18T13:05:52.398 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ OK ] NeoRadosIo.GetXattrs (3001 ms) 2024-06-18T13:05:52.398 INFO:tasks.workunit.client.0.smithi167.stdout: io: [----------] 14 tests from NeoRadosIo (57977 ms total) 2024-06-18T13:05:52.398 INFO:tasks.workunit.client.0.smithi167.stdout: io: 2024-06-18T13:05:52.398 INFO:tasks.workunit.client.0.smithi167.stdout: io: [----------] Global test environment tear-down 2024-06-18T13:05:52.398 INFO:tasks.workunit.client.0.smithi167.stdout: io: [==========] 14 tests from 1 test suite ran. (57977 ms total) 2024-06-18T13:05:52.398 INFO:tasks.workunit.client.0.smithi167.stdout: io: [ PASSED ] 14 tests. 2024-06-18T13:05:52.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:52 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:52.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:52 smithi167 ceph-mon[37701]: osdmap e125: 8 total, 8 up, 8 in 2024-06-18T13:05:52.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:52 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:52.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:52 smithi167 ceph-mon[29496]: osdmap e125: 8 total, 8 up, 8 in 2024-06-18T13:05:52.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:52 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:52 smithi179 ceph-mon[33312]: osdmap e125: 8 total, 8 up, 8 in 2024-06-18T13:05:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: pgmap v139: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 260 KiB/s wr, 8 op/s 2024-06-18T13:05:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]': finished 2024-06-18T13:05:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: osdmap e126: 8 total, 8 up, 8 in 2024-06-18T13:05:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2737825773' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi167-73976-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3688793302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi167-73991-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi167-73976-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi167-73991-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:53.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:53.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi167-73976-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:53.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi167-73991-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:53.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[29496]: osdmap e127: 8 total, 8 up, 8 in 2024-06-18T13:05:53.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: pgmap v139: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 260 KiB/s wr, 8 op/s 2024-06-18T13:05:53.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]': finished 2024-06-18T13:05:53.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: osdmap e126: 8 total, 8 up, 8 in 2024-06-18T13:05:53.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2737825773' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi167-73976-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3688793302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi167-73991-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:53.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi167-73976-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi167-73991-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:53.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:53.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi167-73976-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:53.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi167-73991-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:53.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:53 smithi167 ceph-mon[37701]: osdmap e127: 8 total, 8 up, 8 in 2024-06-18T13:05:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: pgmap v139: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 260 KiB/s wr, 8 op/s 2024-06-18T13:05:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi167-74332-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]': finished 2024-06-18T13:05:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: osdmap e126: 8 total, 8 up, 8 in 2024-06-18T13:05:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2737825773' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi167-73976-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3688793302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi167-73991-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi167-73976-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi167-73991-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi167-73976-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi167-73991-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:53 smithi179 ceph-mon[33312]: osdmap e127: 8 total, 8 up, 8 in 2024-06-18T13:05:54.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:54 smithi167 ceph-mon[29496]: pgmap v142: 492 pgs: 136 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:05:54.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:54.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:54 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:54.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:54 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:05:54.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:54 smithi167 ceph-mon[29496]: osdmap e128: 8 total, 8 up, 8 in 2024-06-18T13:05:54.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:54 smithi167 ceph-mon[37701]: pgmap v142: 492 pgs: 136 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:05:54.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:54.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:54 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:54.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:54 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:05:54.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:54 smithi167 ceph-mon[37701]: osdmap e128: 8 total, 8 up, 8 in 2024-06-18T13:05:54.800 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:54 smithi179 ceph-mon[33312]: pgmap v142: 492 pgs: 136 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:05:54.800 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:54.800 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:54 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:05:54.800 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:54 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:05:54.800 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:54 smithi179 ceph-mon[33312]: osdmap e128: 8 total, 8 up, 8 in 2024-06-18T13:05:55.436 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:05:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:05:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:05:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:55 smithi179 ceph-mon[33312]: osdmap e129: 8 total, 8 up, 8 in 2024-06-18T13:05:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2481640570' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi167-73976-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3285389290' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi167-73991-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:55.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:55 smithi167 ceph-mon[29496]: osdmap e129: 8 total, 8 up, 8 in 2024-06-18T13:05:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2481640570' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi167-73976-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3285389290' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi167-73991-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:55 smithi167 ceph-mon[37701]: osdmap e129: 8 total, 8 up, 8 in 2024-06-18T13:05:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2481640570' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi167-73976-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3285389290' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi167-73991-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:56.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:56 smithi167 ceph-mon[37701]: pgmap v145: 492 pgs: 64 unknown, 32 creating+peering, 396 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.0 KiB/s wr, 6 op/s 2024-06-18T13:05:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2481640570' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi167-73976-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3285389290' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi167-73991-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:56 smithi167 ceph-mon[37701]: osdmap e130: 8 total, 8 up, 8 in 2024-06-18T13:05:56.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:56 smithi167 ceph-mon[29496]: pgmap v145: 492 pgs: 64 unknown, 32 creating+peering, 396 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.0 KiB/s wr, 6 op/s 2024-06-18T13:05:56.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:56.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2481640570' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi167-73976-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:56.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3285389290' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi167-73991-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:56.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:56 smithi167 ceph-mon[29496]: osdmap e130: 8 total, 8 up, 8 in 2024-06-18T13:05:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:56 smithi179 ceph-mon[33312]: pgmap v145: 492 pgs: 64 unknown, 32 creating+peering, 396 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.0 KiB/s wr, 6 op/s 2024-06-18T13:05:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2481640570' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi167-73976-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3285389290' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi167-73991-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:05:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:56 smithi179 ceph-mon[33312]: osdmap e130: 8 total, 8 up, 8 in 2024-06-18T13:05:58.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:58.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:57 smithi167 ceph-mon[37701]: osdmap e131: 8 total, 8 up, 8 in 2024-06-18T13:05:58.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:58.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:57 smithi167 ceph-mon[29496]: osdmap e131: 8 total, 8 up, 8 in 2024-06-18T13:05:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:57 smithi179 ceph-mon[33312]: osdmap e131: 8 total, 8 up, 8 in 2024-06-18T13:05:58.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:05:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[37701]: pgmap v148: 428 pgs: 32 creating+peering, 396 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-18T13:05:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[37701]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[37701]: osdmap e132: 8 total, 8 up, 8 in 2024-06-18T13:05:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-13"}]: dispatch 2024-06-18T13:05:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-13"}]: dispatch 2024-06-18T13:05:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3263811360' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi167-73991-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:59.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi167-73991-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:59.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3112538700' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi167-73976-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:59.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi167-73976-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:59.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[29496]: pgmap v148: 428 pgs: 32 creating+peering, 396 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-18T13:05:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[29496]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[29496]: osdmap e132: 8 total, 8 up, 8 in 2024-06-18T13:05:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-13"}]: dispatch 2024-06-18T13:05:59.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-13"}]: dispatch 2024-06-18T13:05:59.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3263811360' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi167-73991-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:59.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi167-73991-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:59.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3112538700' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi167-73976-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:59.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi167-73976-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:59.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:59.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-mon[33312]: pgmap v148: 428 pgs: 32 creating+peering, 396 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-18T13:05:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:05:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:05:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-mon[33312]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:05:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:05:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-mon[33312]: osdmap e132: 8 total, 8 up, 8 in 2024-06-18T13:05:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-13"}]: dispatch 2024-06-18T13:05:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-13"}]: dispatch 2024-06-18T13:05:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3263811360' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi167-73991-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi167-73991-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3112538700' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi167-73976-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi167-73976-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:05:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:00.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:00.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-13"}]': finished 2024-06-18T13:06:00.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi167-73991-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:00.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi167-73976-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:00.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[37701]: osdmap e133: 8 total, 8 up, 8 in 2024-06-18T13:06:00.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-13", "mode": "writeback"}]: dispatch 2024-06-18T13:06:00.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-13", "mode": "writeback"}]: dispatch 2024-06-18T13:06:00.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:00.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:00.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-13"}]': finished 2024-06-18T13:06:00.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi167-73991-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:00.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi167-73976-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:00.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[29496]: osdmap e133: 8 total, 8 up, 8 in 2024-06-18T13:06:00.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-13", "mode": "writeback"}]: dispatch 2024-06-18T13:06:00.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-13", "mode": "writeback"}]: dispatch 2024-06-18T13:06:00.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:05:59 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:00.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:59 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:00.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-13"}]': finished 2024-06-18T13:06:00.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi167-73991-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:00.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi167-73976-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:00.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:59 smithi179 ceph-mon[33312]: osdmap e133: 8 total, 8 up, 8 in 2024-06-18T13:06:00.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:59 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-13", "mode": "writeback"}]: dispatch 2024-06-18T13:06:00.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-13", "mode": "writeback"}]: dispatch 2024-06-18T13:06:00.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:05:59 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:01.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:00 smithi167 ceph-mon[29496]: pgmap v151: 492 pgs: 32 unknown, 64 creating+peering, 396 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 122 KiB/s wr, 119 op/s 2024-06-18T13:06:01.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:00 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:06:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-13", "mode": "writeback"}]': finished 2024-06-18T13:06:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:00 smithi167 ceph-mon[29496]: osdmap e134: 8 total, 8 up, 8 in 2024-06-18T13:06:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:00 smithi167 ceph-mon[37701]: pgmap v151: 492 pgs: 32 unknown, 64 creating+peering, 396 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 122 KiB/s wr, 119 op/s 2024-06-18T13:06:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:00 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:06:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-13", "mode": "writeback"}]': finished 2024-06-18T13:06:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:00 smithi167 ceph-mon[37701]: osdmap e134: 8 total, 8 up, 8 in 2024-06-18T13:06:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:01.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:00 smithi179 ceph-mon[33312]: pgmap v151: 492 pgs: 32 unknown, 64 creating+peering, 396 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 122 KiB/s wr, 119 op/s 2024-06-18T13:06:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:00 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:06:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-13", "mode": "writeback"}]': finished 2024-06-18T13:06:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:00 smithi179 ceph-mon[33312]: osdmap e134: 8 total, 8 up, 8 in 2024-06-18T13:06:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.0"}]: dispatch 2024-06-18T13:06:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.0"}]: dispatch 2024-06-18T13:06:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.1"}]: dispatch 2024-06-18T13:06:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.1"}]: dispatch 2024-06-18T13:06:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.2"}]: dispatch 2024-06-18T13:06:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.2"}]: dispatch 2024-06-18T13:06:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.3"}]: dispatch 2024-06-18T13:06:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.3"}]: dispatch 2024-06-18T13:06:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.4"}]: dispatch 2024-06-18T13:06:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.4"}]: dispatch 2024-06-18T13:06:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.5"}]: dispatch 2024-06-18T13:06:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.5"}]: dispatch 2024-06-18T13:06:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.6"}]: dispatch 2024-06-18T13:06:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.6"}]: dispatch 2024-06-18T13:06:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.7"}]: dispatch 2024-06-18T13:06:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.7"}]: dispatch 2024-06-18T13:06:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.8"}]: dispatch 2024-06-18T13:06:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.8"}]: dispatch 2024-06-18T13:06:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.9"}]: dispatch 2024-06-18T13:06:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.9"}]: dispatch 2024-06-18T13:06:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: 161.8 scrub starts 2024-06-18T13:06:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: 161.8 scrub ok 2024-06-18T13:06:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: 161.7 scrub starts 2024-06-18T13:06:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: 161.7 scrub ok 2024-06-18T13:06:02.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: osdmap e135: 8 total, 8 up, 8 in 2024-06-18T13:06:02.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1157884259' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi167-73976-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:02.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi167-73976-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:02.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:06:02.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.0"}]: dispatch 2024-06-18T13:06:02.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.0"}]: dispatch 2024-06-18T13:06:02.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.1"}]: dispatch 2024-06-18T13:06:02.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.1"}]: dispatch 2024-06-18T13:06:02.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.2"}]: dispatch 2024-06-18T13:06:02.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.2"}]: dispatch 2024-06-18T13:06:02.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.3"}]: dispatch 2024-06-18T13:06:02.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.3"}]: dispatch 2024-06-18T13:06:02.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.4"}]: dispatch 2024-06-18T13:06:02.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.4"}]: dispatch 2024-06-18T13:06:02.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.5"}]: dispatch 2024-06-18T13:06:02.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.5"}]: dispatch 2024-06-18T13:06:02.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.6"}]: dispatch 2024-06-18T13:06:02.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.6"}]: dispatch 2024-06-18T13:06:02.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.7"}]: dispatch 2024-06-18T13:06:02.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.7"}]: dispatch 2024-06-18T13:06:02.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.8"}]: dispatch 2024-06-18T13:06:02.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.8"}]: dispatch 2024-06-18T13:06:02.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.9"}]: dispatch 2024-06-18T13:06:02.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.9"}]: dispatch 2024-06-18T13:06:02.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: 161.8 scrub starts 2024-06-18T13:06:02.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: 161.8 scrub ok 2024-06-18T13:06:02.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: 161.7 scrub starts 2024-06-18T13:06:02.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: 161.7 scrub ok 2024-06-18T13:06:02.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: osdmap e135: 8 total, 8 up, 8 in 2024-06-18T13:06:02.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1157884259' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi167-73976-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:02.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi167-73976-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:02.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:06:02.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/973986040' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi167-73991-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:02.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:06:02.231 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi167-73991-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:02.232 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:02.232 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: 161.0 scrub starts 2024-06-18T13:06:02.232 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[29496]: 161.0 scrub ok 2024-06-18T13:06:02.232 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/973986040' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi167-73991-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:02.232 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:06:02.232 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi167-73991-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:02.232 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:02.232 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: 161.0 scrub starts 2024-06-18T13:06:02.232 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:01 smithi167 ceph-mon[37701]: 161.0 scrub ok 2024-06-18T13:06:02.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.0"}]: dispatch 2024-06-18T13:06:02.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.0"}]: dispatch 2024-06-18T13:06:02.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.1"}]: dispatch 2024-06-18T13:06:02.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.1"}]: dispatch 2024-06-18T13:06:02.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.2"}]: dispatch 2024-06-18T13:06:02.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.2"}]: dispatch 2024-06-18T13:06:02.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.3"}]: dispatch 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.3"}]: dispatch 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.4"}]: dispatch 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.4"}]: dispatch 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.5"}]: dispatch 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.5"}]: dispatch 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.6"}]: dispatch 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.6"}]: dispatch 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.7"}]: dispatch 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.7"}]: dispatch 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.8"}]: dispatch 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.8"}]: dispatch 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.9"}]: dispatch 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.9"}]: dispatch 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: 161.8 scrub starts 2024-06-18T13:06:02.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: 161.8 scrub ok 2024-06-18T13:06:02.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: 161.7 scrub starts 2024-06-18T13:06:02.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: 161.7 scrub ok 2024-06-18T13:06:02.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: osdmap e135: 8 total, 8 up, 8 in 2024-06-18T13:06:02.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1157884259' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi167-73976-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:02.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi167-73976-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:02.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:06:02.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/973986040' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi167-73991-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:02.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:06:02.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi167-73991-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:02.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:02.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: 161.0 scrub starts 2024-06-18T13:06:02.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:01 smithi179 ceph-mon[33312]: 161.0 scrub ok 2024-06-18T13:06:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: pgmap v154: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 114 KiB/s wr, 117 op/s 2024-06-18T13:06:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: 161.5 deep-scrub starts 2024-06-18T13:06:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: 161.5 deep-scrub ok 2024-06-18T13:06:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: 161.1 scrub starts 2024-06-18T13:06:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: 161.1 scrub ok 2024-06-18T13:06:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: 161.6 scrub starts 2024-06-18T13:06:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: 161.6 scrub ok 2024-06-18T13:06:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: 161.9 scrub starts 2024-06-18T13:06:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: 161.9 scrub ok 2024-06-18T13:06:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: 161.2 scrub starts 2024-06-18T13:06:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: 161.2 scrub ok 2024-06-18T13:06:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi167-73976-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]': finished 2024-06-18T13:06:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi167-73991-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: osdmap e136: 8 total, 8 up, 8 in 2024-06-18T13:06:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:06:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:06:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: 161.3 scrub starts 2024-06-18T13:06:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[37701]: 161.3 scrub ok 2024-06-18T13:06:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: pgmap v154: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 114 KiB/s wr, 117 op/s 2024-06-18T13:06:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: 161.5 deep-scrub starts 2024-06-18T13:06:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: 161.5 deep-scrub ok 2024-06-18T13:06:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: 161.1 scrub starts 2024-06-18T13:06:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: 161.1 scrub ok 2024-06-18T13:06:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: 161.6 scrub starts 2024-06-18T13:06:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: 161.6 scrub ok 2024-06-18T13:06:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: 161.9 scrub starts 2024-06-18T13:06:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: 161.9 scrub ok 2024-06-18T13:06:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: 161.2 scrub starts 2024-06-18T13:06:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: 161.2 scrub ok 2024-06-18T13:06:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi167-73976-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]': finished 2024-06-18T13:06:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi167-73991-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:03.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: osdmap e136: 8 total, 8 up, 8 in 2024-06-18T13:06:03.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:06:03.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:06:03.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:03.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: 161.3 scrub starts 2024-06-18T13:06:03.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:02 smithi167 ceph-mon[29496]: 161.3 scrub ok 2024-06-18T13:06:03.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: pgmap v154: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 114 KiB/s wr, 117 op/s 2024-06-18T13:06:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: 161.5 deep-scrub starts 2024-06-18T13:06:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: 161.5 deep-scrub ok 2024-06-18T13:06:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: 161.1 scrub starts 2024-06-18T13:06:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: 161.1 scrub ok 2024-06-18T13:06:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: 161.6 scrub starts 2024-06-18T13:06:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: 161.6 scrub ok 2024-06-18T13:06:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: 161.9 scrub starts 2024-06-18T13:06:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: 161.9 scrub ok 2024-06-18T13:06:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: 161.2 scrub starts 2024-06-18T13:06:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: 161.2 scrub ok 2024-06-18T13:06:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi167-73976-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]': finished 2024-06-18T13:06:03.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi167-73991-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:03.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: osdmap e136: 8 total, 8 up, 8 in 2024-06-18T13:06:03.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1098532507' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:06:03.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]: dispatch 2024-06-18T13:06:03.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:03.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: 161.3 scrub starts 2024-06-18T13:06:03.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:02 smithi179 ceph-mon[33312]: 161.3 scrub ok 2024-06-18T13:06:03.424 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: Running main() from gmock_main.cc 2024-06-18T13:06:03.424 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-06-18T13:06:03.424 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [----------] Global test environment set-up. 2024-06-18T13:06:03.424 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-06-18T13:06:03.424 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-06-18T13:06:03.425 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (2553 ms) 2024-06-18T13:06:03.425 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-06-18T13:06:03.425 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (2013 ms) 2024-06-18T13:06:03.425 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-06-18T13:06:03.425 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (3273 ms) 2024-06-18T13:06:03.425 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-06-18T13:06:03.425 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2300 ms) 2024-06-18T13:06:03.425 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (10139 ms total) 2024-06-18T13:06:03.425 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: 2024-06-18T13:06:03.425 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-06-18T13:06:03.425 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-06-18T13:06:03.425 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (5645 ms) 2024-06-18T13:06:03.425 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-06-18T13:06:03.425 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4781 ms) 2024-06-18T13:06:03.426 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-06-18T13:06:03.426 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (6663 ms) 2024-06-18T13:06:03.426 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (17090 ms total) 2024-06-18T13:06:03.426 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: 2024-06-18T13:06:03.426 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-06-18T13:06:03.426 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-06-18T13:06:03.426 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3133 ms) 2024-06-18T13:06:03.426 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-06-18T13:06:03.426 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (1853 ms) 2024-06-18T13:06:03.426 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-06-18T13:06:03.426 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2009 ms) 2024-06-18T13:06:03.426 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-06-18T13:06:03.426 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2007 ms) 2024-06-18T13:06:03.427 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9002 ms total) 2024-06-18T13:06:03.427 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: 2024-06-18T13:06:03.427 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-06-18T13:06:03.427 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-06-18T13:06:03.427 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4030 ms) 2024-06-18T13:06:03.427 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-06-18T13:06:03.427 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4008 ms) 2024-06-18T13:06:03.427 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8038 ms total) 2024-06-18T13:06:03.427 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: 2024-06-18T13:06:03.427 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [----------] Global test environment tear-down 2024-06-18T13:06:03.427 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (69269 ms total) 2024-06-18T13:06:03.427 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-06-18T13:06:04.433 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:04 smithi179 ceph-mon[33312]: 161.4 scrub starts 2024-06-18T13:06:04.433 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:04 smithi179 ceph-mon[33312]: 161.4 scrub ok 2024-06-18T13:06:04.433 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]': finished 2024-06-18T13:06:04.433 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:04 smithi179 ceph-mon[33312]: osdmap e137: 8 total, 8 up, 8 in 2024-06-18T13:06:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:04 smithi167 ceph-mon[37701]: 161.4 scrub starts 2024-06-18T13:06:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:04 smithi167 ceph-mon[37701]: 161.4 scrub ok 2024-06-18T13:06:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]': finished 2024-06-18T13:06:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:04 smithi167 ceph-mon[37701]: osdmap e137: 8 total, 8 up, 8 in 2024-06-18T13:06:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:04 smithi167 ceph-mon[37701]: pgmap v157: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:06:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:04 smithi167 ceph-mon[29496]: 161.4 scrub starts 2024-06-18T13:06:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:04 smithi167 ceph-mon[29496]: 161.4 scrub ok 2024-06-18T13:06:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi167-74332-15"}]': finished 2024-06-18T13:06:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:04 smithi167 ceph-mon[29496]: osdmap e137: 8 total, 8 up, 8 in 2024-06-18T13:06:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:04 smithi167 ceph-mon[29496]: pgmap v157: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:06:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:04.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:04 smithi179 ceph-mon[33312]: pgmap v157: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:06:04.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:05.433 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:06:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:06:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[37701]: osdmap e138: 8 total, 8 up, 8 in 2024-06-18T13:06:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1072804970' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi167-73991-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi167-73991-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/519929736' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi167-73976-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi167-73976-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi167-73991-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi167-73976-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[37701]: osdmap e139: 8 total, 8 up, 8 in 2024-06-18T13:06:05.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:05.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[29496]: osdmap e138: 8 total, 8 up, 8 in 2024-06-18T13:06:05.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1072804970' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi167-73991-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:05.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi167-73991-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:05.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/519929736' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi167-73976-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:05.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi167-73976-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:05.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:05.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi167-73991-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:05.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi167-73976-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:05.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:05 smithi167 ceph-mon[29496]: osdmap e139: 8 total, 8 up, 8 in 2024-06-18T13:06:05.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:05 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:05 smithi179 ceph-mon[33312]: osdmap e138: 8 total, 8 up, 8 in 2024-06-18T13:06:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1072804970' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi167-73991-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi167-73991-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/519929736' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi167-73976-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi167-73976-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi167-73991-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi167-73976-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:05 smithi179 ceph-mon[33312]: osdmap e139: 8 total, 8 up, 8 in 2024-06-18T13:06:06.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:06 smithi167 ceph-mon[37701]: pgmap v160: 484 pgs: 64 unknown, 32 creating+peering, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 57 KiB/s rd, 76 op/s 2024-06-18T13:06:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-18T13:06:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:06 smithi167 ceph-mon[37701]: osdmap e140: 8 total, 8 up, 8 in 2024-06-18T13:06:06.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:06 smithi167 ceph-mon[29496]: pgmap v160: 484 pgs: 64 unknown, 32 creating+peering, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 57 KiB/s rd, 76 op/s 2024-06-18T13:06:06.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:06.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-18T13:06:06.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:06.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:06.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:06.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:06 smithi167 ceph-mon[29496]: osdmap e140: 8 total, 8 up, 8 in 2024-06-18T13:06:06.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:06 smithi179 ceph-mon[33312]: pgmap v160: 484 pgs: 64 unknown, 32 creating+peering, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 57 KiB/s rd, 76 op/s 2024-06-18T13:06:06.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-18T13:06:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:06 smithi179 ceph-mon[33312]: osdmap e140: 8 total, 8 up, 8 in 2024-06-18T13:06:07.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app2"}]: dispatch 2024-06-18T13:06:07.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:07.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:07.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:07.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:07.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[29496]: osdmap e141: 8 total, 8 up, 8 in 2024-06-18T13:06:07.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:07.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3027145608' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73976-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:07.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app2"}]: dispatch 2024-06-18T13:06:07.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:07.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:07.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:07.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:07.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[37701]: osdmap e141: 8 total, 8 up, 8 in 2024-06-18T13:06:07.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:07.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3027145608' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73976-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app2"}]: dispatch 2024-06-18T13:06:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi167-74155-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:07 smithi179 ceph-mon[33312]: osdmap e141: 8 total, 8 up, 8 in 2024-06-18T13:06:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3027145608' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73976-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:08.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[37701]: pgmap v163: 484 pgs: 64 unknown, 32 creating+peering, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 57 KiB/s rd, 76 op/s 2024-06-18T13:06:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1760188893' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-06-18T13:06:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-18T13:06:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-18T13:06:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3027145608' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73976-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-06-18T13:06:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[37701]: osdmap e142: 8 total, 8 up, 8 in 2024-06-18T13:06:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-18T13:06:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-18T13:06:08.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[29496]: pgmap v163: 484 pgs: 64 unknown, 32 creating+peering, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 57 KiB/s rd, 76 op/s 2024-06-18T13:06:08.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1760188893' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:08.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-06-18T13:06:08.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-18T13:06:08.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-18T13:06:08.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:08.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:08.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3027145608' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73976-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:08.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-06-18T13:06:08.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[29496]: osdmap e142: 8 total, 8 up, 8 in 2024-06-18T13:06:08.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-18T13:06:08.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-18T13:06:08.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:06:08 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:06:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:08 smithi179 ceph-mon[33312]: pgmap v163: 484 pgs: 64 unknown, 32 creating+peering, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 57 KiB/s rd, 76 op/s 2024-06-18T13:06:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1760188893' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-06-18T13:06:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-18T13:06:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-18T13:06:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3027145608' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73976-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-06-18T13:06:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:08 smithi179 ceph-mon[33312]: osdmap e142: 8 total, 8 up, 8 in 2024-06-18T13:06:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-18T13:06:08.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-18T13:06:09.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:09 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:06:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:06:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-06-18T13:06:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:09 smithi167 ceph-mon[37701]: osdmap e143: 8 total, 8 up, 8 in 2024-06-18T13:06:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:09 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:06:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:06:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-06-18T13:06:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:09 smithi167 ceph-mon[29496]: osdmap e143: 8 total, 8 up, 8 in 2024-06-18T13:06:09.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:09 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:06:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:06:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-06-18T13:06:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:09 smithi179 ceph-mon[33312]: osdmap e143: 8 total, 8 up, 8 in 2024-06-18T13:06:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[37701]: pgmap v165: 452 pgs: 1 active, 25 creating+activating, 426 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:06:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1"}]: dispatch 2024-06-18T13:06:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2797928685' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74374-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1"}]: dispatch 2024-06-18T13:06:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74374-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1"}]': finished 2024-06-18T13:06:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74374-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[37701]: osdmap e144: 8 total, 8 up, 8 in 2024-06-18T13:06:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1218599884' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi167-73976-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[29496]: pgmap v165: 452 pgs: 1 active, 25 creating+activating, 426 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:06:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1"}]: dispatch 2024-06-18T13:06:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2797928685' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74374-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1"}]: dispatch 2024-06-18T13:06:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74374-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1"}]': finished 2024-06-18T13:06:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74374-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[29496]: osdmap e144: 8 total, 8 up, 8 in 2024-06-18T13:06:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1218599884' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi167-73976-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:10 smithi179 ceph-mon[33312]: pgmap v165: 452 pgs: 1 active, 25 creating+activating, 426 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:06:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2694852699' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1"}]: dispatch 2024-06-18T13:06:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2797928685' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74374-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1"}]: dispatch 2024-06-18T13:06:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74374-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:10 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi167-74155-1","app":"app1","key":"key1"}]': finished 2024-06-18T13:06:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74374-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:10 smithi179 ceph-mon[33312]: osdmap e144: 8 total, 8 up, 8 in 2024-06-18T13:06:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1218599884' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi167-73976-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:11.448 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-06-18T13:06:11.449 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-06-18T13:06:11.449 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-06-18T13:06:11.449 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-06-18T13:06:11.449 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-06-18T13:06:11.449 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-06-18T13:06:11.449 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: 2024-06-18T13:06:11.449 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-06-18T13:06:11.449 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: seed 74155 2024-06-18T13:06:11.449 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-06-18T13:06:11.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (0 ms) 2024-06-18T13:06:11.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-06-18T13:06:11.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (120 ms) 2024-06-18T13:06:11.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-06-18T13:06:11.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (5 ms) 2024-06-18T13:06:11.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-06-18T13:06:11.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (5 ms) 2024-06-18T13:06:11.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-06-18T13:06:11.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (5 ms) 2024-06-18T13:06:11.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-06-18T13:06:11.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-06-18T13:06:11.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-06-18T13:06:11.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (1 ms) 2024-06-18T13:06:11.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-06-18T13:06:11.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (4 ms) 2024-06-18T13:06:11.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-06-18T13:06:11.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (1 ms) 2024-06-18T13:06:11.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-06-18T13:06:11.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (7 ms) 2024-06-18T13:06:11.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-06-18T13:06:11.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-06-18T13:06:11.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-06-18T13:06:11.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (3 ms) 2024-06-18T13:06:11.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-06-18T13:06:11.451 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (6 ms) 2024-06-18T13:06:11.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-06-18T13:06:11.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: osd_max_attr_size = 0 2024-06-18T13:06:11.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-06-18T13:06:11.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1622 ms) 2024-06-18T13:06:11.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-06-18T13:06:11.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (385 ms) 2024-06-18T13:06:11.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-06-18T13:06:11.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-06-18T13:06:11.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: done waiting, doing copies 2024-06-18T13:06:11.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: waiting for final deep scrubs... 2024-06-18T13:06:11.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: done waiting 2024-06-18T13:06:11.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60887 ms) 2024-06-18T13:06:11.452 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-06-18T13:06:11.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (4 ms) 2024-06-18T13:06:11.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-06-18T13:06:11.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (2 ms) 2024-06-18T13:06:11.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-06-18T13:06:11.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4635 ms) 2024-06-18T13:06:11.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-06-18T13:06:11.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-06-18T13:06:11.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-06-18T13:06:11.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-06-18T13:06:11.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-06-18T13:06:11.453 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-06-18T13:06:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1770362787' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi167-73991-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi167-73991-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:11.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1218599884' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi167-73976-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:11.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi167-73991-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:11.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:11 smithi167 ceph-mon[37701]: osdmap e145: 8 total, 8 up, 8 in 2024-06-18T13:06:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1770362787' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi167-73991-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi167-73991-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1218599884' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi167-73976-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi167-73991-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:11 smithi167 ceph-mon[29496]: osdmap e145: 8 total, 8 up, 8 in 2024-06-18T13:06:11.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1770362787' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi167-73991-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi167-73991-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1218599884' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi167-73976-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi167-73991-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:11 smithi179 ceph-mon[33312]: osdmap e145: 8 total, 8 up, 8 in 2024-06-18T13:06:13.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[29496]: pgmap v168: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:06:13.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74155-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:13.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74155-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:13.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:13.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74155-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:13.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[29496]: osdmap e146: 8 total, 8 up, 8 in 2024-06-18T13:06:13.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74155-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74155-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:13.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[37701]: pgmap v168: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:06:13.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74155-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:13.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74155-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:13.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:13.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74155-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:13.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[37701]: osdmap e146: 8 total, 8 up, 8 in 2024-06-18T13:06:13.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74155-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74155-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:12 smithi179 ceph-mon[33312]: pgmap v168: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:06:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74155-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74155-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74155-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:12 smithi179 ceph-mon[33312]: osdmap e146: 8 total, 8 up, 8 in 2024-06-18T13:06:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74155-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74155-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:13.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:14.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:14 smithi179 ceph-mon[33312]: osdmap e147: 8 total, 8 up, 8 in 2024-06-18T13:06:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/604778045' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi167-73991-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4016982894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi167-73976-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi167-73991-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:14.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[29496]: osdmap e147: 8 total, 8 up, 8 in 2024-06-18T13:06:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/604778045' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi167-73991-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4016982894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi167-73976-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi167-73991-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[37701]: osdmap e147: 8 total, 8 up, 8 in 2024-06-18T13:06:14.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/604778045' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi167-73991-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:14.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4016982894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi167-73976-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:14.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi167-73991-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:14.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:14.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:14.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:14.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:15.387 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:06:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:06:15.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[29496]: pgmap v172: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:06:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74155-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74155-24"}]': finished 2024-06-18T13:06:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4016982894' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi167-73976-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi167-73991-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[29496]: osdmap e148: 8 total, 8 up, 8 in 2024-06-18T13:06:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi167-74374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[37701]: pgmap v172: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:06:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74155-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74155-24"}]': finished 2024-06-18T13:06:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4016982894' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi167-73976-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:15.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi167-73991-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:15.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:15.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[37701]: osdmap e148: 8 total, 8 up, 8 in 2024-06-18T13:06:15.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi167-74374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:15.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:15 smithi179 ceph-mon[33312]: pgmap v172: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-06-18T13:06:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74155-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74155-24"}]': finished 2024-06-18T13:06:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4016982894' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi167-73976-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi167-73991-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:15 smithi179 ceph-mon[33312]: osdmap e148: 8 total, 8 up, 8 in 2024-06-18T13:06:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi167-74374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:16.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:16 smithi167 ceph-mon[37701]: pgmap v175: 396 pgs: 40 unknown, 356 active+clean; 592 KiB data, 543 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-18T13:06:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:16 smithi167 ceph-mon[37701]: osdmap e149: 8 total, 8 up, 8 in 2024-06-18T13:06:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:16 smithi167 ceph-mon[29496]: pgmap v175: 396 pgs: 40 unknown, 356 active+clean; 592 KiB data, 543 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-18T13:06:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:16 smithi167 ceph-mon[29496]: osdmap e149: 8 total, 8 up, 8 in 2024-06-18T13:06:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:16.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:16 smithi179 ceph-mon[33312]: pgmap v175: 396 pgs: 40 unknown, 356 active+clean; 592 KiB data, 543 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-18T13:06:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:16 smithi179 ceph-mon[33312]: osdmap e149: 8 total, 8 up, 8 in 2024-06-18T13:06:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:17.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi167-74374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]': finished 2024-06-18T13:06:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[37701]: osdmap e150: 8 total, 8 up, 8 in 2024-06-18T13:06:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1709018579' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi167-73991-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1143918571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi167-73976-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[37701]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:17.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi167-74374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]': finished 2024-06-18T13:06:17.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[29496]: osdmap e150: 8 total, 8 up, 8 in 2024-06-18T13:06:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1709018579' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi167-73991-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1143918571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi167-73976-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:17 smithi167 ceph-mon[29496]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi167-74374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]': finished 2024-06-18T13:06:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:17 smithi179 ceph-mon[33312]: osdmap e150: 8 total, 8 up, 8 in 2024-06-18T13:06:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1709018579' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi167-73991-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1143918571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi167-73976-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:17 smithi179 ceph-mon[33312]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:18.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1709018579' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi167-73991-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1143918571' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi167-73976-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:18 smithi167 ceph-mon[37701]: osdmap e151: 8 total, 8 up, 8 in 2024-06-18T13:06:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:18 smithi167 ceph-mon[37701]: pgmap v178: 468 pgs: 112 unknown, 356 active+clean; 592 KiB data, 543 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:06:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1709018579' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi167-73991-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1143918571' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi167-73976-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:18 smithi167 ceph-mon[29496]: osdmap e151: 8 total, 8 up, 8 in 2024-06-18T13:06:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:18 smithi167 ceph-mon[29496]: pgmap v178: 468 pgs: 112 unknown, 356 active+clean; 592 KiB data, 543 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:06:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:18.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1709018579' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi167-73991-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1143918571' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi167-73976-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74155-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:18 smithi179 ceph-mon[33312]: osdmap e151: 8 total, 8 up, 8 in 2024-06-18T13:06:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:18 smithi179 ceph-mon[33312]: pgmap v178: 468 pgs: 112 unknown, 356 active+clean; 592 KiB data, 543 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:06:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:18.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:06:18 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:06:19.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:19 smithi167 ceph-mon[37701]: osdmap e152: 8 total, 8 up, 8 in 2024-06-18T13:06:19.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:19 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:19 smithi167 ceph-mon[29496]: osdmap e152: 8 total, 8 up, 8 in 2024-06-18T13:06:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:19 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:19.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:19 smithi179 ceph-mon[33312]: osdmap e152: 8 total, 8 up, 8 in 2024-06-18T13:06:19.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:19 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[37701]: osdmap e153: 8 total, 8 up, 8 in 2024-06-18T13:06:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[37701]: pgmap v181: 428 pgs: 64 unknown, 364 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:06:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1608763132' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi167-73976-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi167-73976-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/618125966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi167-73991-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]': finished 2024-06-18T13:06:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi167-73976-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:20.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/618125966' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi167-73991-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:20.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[37701]: osdmap e154: 8 total, 8 up, 8 in 2024-06-18T13:06:20.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:20.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:20.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[29496]: osdmap e153: 8 total, 8 up, 8 in 2024-06-18T13:06:20.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[29496]: pgmap v181: 428 pgs: 64 unknown, 364 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:06:20.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:20.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1608763132' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi167-73976-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:20.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:20.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi167-73976-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:20.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/618125966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi167-73991-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:20.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:20.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]': finished 2024-06-18T13:06:20.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi167-73976-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:20.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/618125966' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi167-73991-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:20.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[29496]: osdmap e154: 8 total, 8 up, 8 in 2024-06-18T13:06:20.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:20.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:20 smithi179 ceph-mon[33312]: osdmap e153: 8 total, 8 up, 8 in 2024-06-18T13:06:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:20 smithi179 ceph-mon[33312]: pgmap v181: 428 pgs: 64 unknown, 364 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:06:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1608763132' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi167-73976-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:20.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:20.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi167-73976-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:20.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/618125966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi167-73991-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:20.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:20.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74155-24"}]': finished 2024-06-18T13:06:20.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi167-73976-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:20.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/618125966' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi167-73991-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:20.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:20 smithi179 ceph-mon[33312]: osdmap e154: 8 total, 8 up, 8 in 2024-06-18T13:06:20.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4214888457' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:20.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]: dispatch 2024-06-18T13:06:21.436 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP (67 api_snapshots_pp: Running main() from gmock_main.cc 2024-06-18T13:06:21.436 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-06-18T13:06:21.437 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-06-18T13:06:21.437 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-06-18T13:06:21.437 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: seed 74374 2024-06-18T13:06:21.437 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-06-18T13:06:21.437 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (5325 ms) 2024-06-18T13:06:21.437 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-06-18T13:06:21.437 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (4043 ms) 2024-06-18T13:06:21.437 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-06-18T13:06:21.437 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2795 ms) 2024-06-18T13:06:21.437 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-06-18T13:06:21.437 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (3077 ms) 2024-06-18T13:06:21.438 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-06-18T13:06:21.438 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (5463 ms) 2024-06-18T13:06:21.438 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (20703 ms total) 2024-06-18T13:06:21.438 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: 2024-06-18T13:06:21.438 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-06-18T13:06:21.438 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-06-18T13:06:21.438 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (5089 ms) 2024-06-18T13:06:21.438 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-06-18T13:06:21.438 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (4883 ms) 2024-06-18T13:06:21.438 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-06-18T13:06:21.438 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (7024 ms) 2024-06-18T13:06:21.438 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-06-18T13:06:21.438 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3992 ms) 2024-06-18T13:06:21.439 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-06-18T13:06:21.439 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2088 ms) 2024-06-18T13:06:21.439 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-06-18T13:06:21.439 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3124-g3781e3a5/rpm/el9/BUILD/ceph-19.0.0-3124-g3781e3a5/src/test/librados/snapshots_cxx.cc:461: Skipped 2024-06-18T13:06:21.439 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: 2024-06-18T13:06:21.439 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-06-18T13:06:21.439 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-06-18T13:06:21.439 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi167-74374-7 2024-06-18T13:06:21.439 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: waiting for snaps to purge 2024-06-18T13:06:21.439 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17931 ms) 2024-06-18T13:06:21.439 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (41007 ms total) 2024-06-18T13:06:21.439 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: 2024-06-18T13:06:21.439 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-06-18T13:06:21.440 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-06-18T13:06:21.440 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (9 ms) 2024-06-18T13:06:21.440 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-06-18T13:06:21.440 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5192 ms) 2024-06-18T13:06:21.440 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5201 ms total) 2024-06-18T13:06:21.440 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: 2024-06-18T13:06:21.440 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-06-18T13:06:21.440 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-06-18T13:06:21.440 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3025 ms) 2024-06-18T13:06:21.440 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-06-18T13:06:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]': finished 2024-06-18T13:06:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:21 smithi167 ceph-mon[37701]: osdmap e155: 8 total, 8 up, 8 in 2024-06-18T13:06:21.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:21.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]': finished 2024-06-18T13:06:21.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:21 smithi167 ceph-mon[29496]: osdmap e155: 8 total, 8 up, 8 in 2024-06-18T13:06:21.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74155-24"}]': finished 2024-06-18T13:06:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:21 smithi179 ceph-mon[33312]: osdmap e155: 8 total, 8 up, 8 in 2024-06-18T13:06:22.450 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnaps snapshots: Running main() from gmock_main.cc 2024-06-18T13:06:22.450 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-06-18T13:06:22.450 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [----------] Global test environment set-up. 2024-06-18T13:06:22.450 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-06-18T13:06:22.450 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (12573 ms) 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (6834 ms) 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (5127 ms) 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (5431 ms) 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (7935 ms) 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (37900 ms total) 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (6040 ms) 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-06-18T13:06:22.451 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (5954 ms) 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (8010 ms) 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (6000 ms) 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4023 ms) 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi167-75312-11. 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: Waiting for snaps to purge. 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (20025 ms) 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (50052 ms total) 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [----------] Global test environment tear-down 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (87953 ms total) 2024-06-18T13:06:22.452 INFO:tasks.workunit.client.0.smithi167.stdout: snapshots: [ PASSED ] 11 tests. 2024-06-18T13:06:22.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[37701]: pgmap v184: 364 pgs: 364 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:06:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[37701]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[37701]: osdmap e156: 8 total, 8 up, 8 in 2024-06-18T13:06:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1817789370' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi167-74155-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/86289613' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi167-73976-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi167-74155-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi167-73976-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1947224963' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73991-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:22.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[29496]: pgmap v184: 364 pgs: 364 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:06:22.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:22.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[29496]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:22.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[29496]: osdmap e156: 8 total, 8 up, 8 in 2024-06-18T13:06:22.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1817789370' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi167-74155-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:22.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/86289613' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi167-73976-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:22.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi167-74155-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:22.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi167-73976-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:22.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1947224963' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73991-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:22.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:22 smithi179 ceph-mon[33312]: pgmap v184: 364 pgs: 364 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:06:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:22 smithi179 ceph-mon[33312]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:22 smithi179 ceph-mon[33312]: osdmap e156: 8 total, 8 up, 8 in 2024-06-18T13:06:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1817789370' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi167-74155-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/86289613' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi167-73976-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi167-74155-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:22.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi167-73976-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:22.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1947224963' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73991-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73991-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]: dispatch 2024-06-18T13:06:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]: dispatch 2024-06-18T13:06:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi167-74155-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi167-73976-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73991-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]': finished 2024-06-18T13:06:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[37701]: osdmap e157: 8 total, 8 up, 8 in 2024-06-18T13:06:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:23.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-18T13:06:23.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-18T13:06:23.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73991-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:23.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:23.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]: dispatch 2024-06-18T13:06:23.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]: dispatch 2024-06-18T13:06:23.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi167-74155-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:23.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi167-73976-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:23.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73991-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:23.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]': finished 2024-06-18T13:06:23.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[29496]: osdmap e157: 8 total, 8 up, 8 in 2024-06-18T13:06:23.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:23.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:23.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-18T13:06:23.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-18T13:06:23.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73991-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:23.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:23.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]: dispatch 2024-06-18T13:06:23.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]: dispatch 2024-06-18T13:06:23.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi167-74155-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:23.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi167-73976-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:23.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi167-73991-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:23.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]': finished 2024-06-18T13:06:23.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:23 smithi179 ceph-mon[33312]: osdmap e157: 8 total, 8 up, 8 in 2024-06-18T13:06:23.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:23.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:23.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-18T13:06:23.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-18T13:06:24.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:24 smithi167 ceph-mon[37701]: pgmap v186: 428 pgs: 96 unknown, 332 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:24 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:06:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]': finished 2024-06-18T13:06:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:24 smithi167 ceph-mon[37701]: osdmap e158: 8 total, 8 up, 8 in 2024-06-18T13:06:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:24 smithi167 ceph-mon[29496]: pgmap v186: 428 pgs: 96 unknown, 332 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:24 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:06:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]': finished 2024-06-18T13:06:24.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:24 smithi167 ceph-mon[29496]: osdmap e158: 8 total, 8 up, 8 in 2024-06-18T13:06:24.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:24.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:24 smithi179 ceph-mon[33312]: pgmap v186: 428 pgs: 96 unknown, 332 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:24.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:24 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:06:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]': finished 2024-06-18T13:06:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:24 smithi179 ceph-mon[33312]: osdmap e158: 8 total, 8 up, 8 in 2024-06-18T13:06:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:25.471 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:06:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:06:25.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-06-18T13:06:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[29496]: osdmap e159: 8 total, 8 up, 8 in 2024-06-18T13:06:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3445801559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi167-73976-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/435171908' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi167-74155-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/391852894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi167-73991-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi167-73976-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi167-74155-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi167-73991-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-06-18T13:06:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[37701]: osdmap e159: 8 total, 8 up, 8 in 2024-06-18T13:06:25.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3445801559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi167-73976-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/435171908' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi167-74155-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/391852894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi167-73991-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi167-73976-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi167-74155-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi167-73991-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-06-18T13:06:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:25 smithi179 ceph-mon[33312]: osdmap e159: 8 total, 8 up, 8 in 2024-06-18T13:06:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3445801559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi167-73976-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/435171908' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi167-74155-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/391852894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi167-73991-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi167-73976-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi167-74155-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi167-73991-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[29496]: pgmap v189: 332 pgs: 1 active+clean+snaptrim, 331 active+clean; 592 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 766 B/s wr, 2 op/s 2024-06-18T13:06:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-06-18T13:06:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi167-73976-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi167-74155-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi167-73991-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[29496]: osdmap e160: 8 total, 8 up, 8 in 2024-06-18T13:06:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[37701]: pgmap v189: 332 pgs: 1 active+clean+snaptrim, 331 active+clean; 592 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 766 B/s wr, 2 op/s 2024-06-18T13:06:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-06-18T13:06:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi167-73976-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi167-74155-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi167-73991-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[37701]: osdmap e160: 8 total, 8 up, 8 in 2024-06-18T13:06:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:26.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:26 smithi179 ceph-mon[33312]: pgmap v189: 332 pgs: 1 active+clean+snaptrim, 331 active+clean; 592 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 766 B/s wr, 2 op/s 2024-06-18T13:06:26.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:26 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-06-18T13:06:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi167-73976-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi167-74155-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi167-73991-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:26 smithi179 ceph-mon[33312]: osdmap e160: 8 total, 8 up, 8 in 2024-06-18T13:06:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:27.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:27.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:27 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]': finished 2024-06-18T13:06:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:27 smithi179 ceph-mon[33312]: osdmap e161: 8 total, 8 up, 8 in 2024-06-18T13:06:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:27.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:27.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:27 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:27.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]': finished 2024-06-18T13:06:27.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:27 smithi167 ceph-mon[29496]: osdmap e161: 8 total, 8 up, 8 in 2024-06-18T13:06:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:27 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi167-74374-16"}]': finished 2024-06-18T13:06:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:27 smithi167 ceph-mon[37701]: osdmap e161: 8 total, 8 up, 8 in 2024-06-18T13:06:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi167-74374-16"}]: dispatch 2024-06-18T13:06:28.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:06:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: pgmap v192: 420 pgs: 96 unknown, 324 active+clean; 592 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:06:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi167-73976-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi167-73976-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi167-74374-16"}]': finished 2024-06-18T13:06:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi167-73976-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi167-73976-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: osdmap e162: 8 total, 8 up, 8 in 2024-06-18T13:06:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi167-73976-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1643080176' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi167-73991-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1437876477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi167-74155-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi167-73991-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:28.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: pgmap v192: 420 pgs: 96 unknown, 324 active+clean; 592 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:06:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi167-73976-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi167-73976-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi167-74374-16"}]': finished 2024-06-18T13:06:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi167-73976-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi167-73976-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: osdmap e162: 8 total, 8 up, 8 in 2024-06-18T13:06:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi167-73976-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1643080176' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi167-73991-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1437876477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi167-74155-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi167-73991-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:28.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: pgmap v192: 420 pgs: 96 unknown, 324 active+clean; 592 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:06:28.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi167-73976-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:28.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:28.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi167-73976-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:28.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/600940468' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi167-74374-16"}]': finished 2024-06-18T13:06:28.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi167-73976-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:28.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi167-73976-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: osdmap e162: 8 total, 8 up, 8 in 2024-06-18T13:06:28.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi167-73976-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:28.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1643080176' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi167-73991-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:28.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1437876477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi167-74155-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:28.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi167-73991-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:28.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:29.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:29 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:29 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-06-18T13:06:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1437876477' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi167-74155-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:29 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:29 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-06-18T13:06:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1437876477' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi167-74155-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:30.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:29 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:29 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-06-18T13:06:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1437876477' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi167-74155-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:30.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[37701]: pgmap v195: 388 pgs: 64 creating+peering, 324 active+clean; 592 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.7 KiB/s wr, 7 op/s; 31 B/s, 0 objects/s recovering 2024-06-18T13:06:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi167-73991-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-06-18T13:06:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[37701]: osdmap e163: 8 total, 8 up, 8 in 2024-06-18T13:06:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi167-73976-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi167-73976-27"}]': finished 2024-06-18T13:06:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[37701]: osdmap e164: 8 total, 8 up, 8 in 2024-06-18T13:06:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[29496]: pgmap v195: 388 pgs: 64 creating+peering, 324 active+clean; 592 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.7 KiB/s wr, 7 op/s; 31 B/s, 0 objects/s recovering 2024-06-18T13:06:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi167-73991-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-06-18T13:06:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[29496]: osdmap e163: 8 total, 8 up, 8 in 2024-06-18T13:06:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi167-73976-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi167-73976-27"}]': finished 2024-06-18T13:06:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[29496]: osdmap e164: 8 total, 8 up, 8 in 2024-06-18T13:06:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:30 smithi179 ceph-mon[33312]: pgmap v195: 388 pgs: 64 creating+peering, 324 active+clean; 592 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.7 KiB/s wr, 7 op/s; 31 B/s, 0 objects/s recovering 2024-06-18T13:06:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi167-73991-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:30 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-06-18T13:06:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:30 smithi179 ceph-mon[33312]: osdmap e163: 8 total, 8 up, 8 in 2024-06-18T13:06:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi167-73976-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi167-73976-27"}]': finished 2024-06-18T13:06:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:30 smithi179 ceph-mon[33312]: osdmap e164: 8 total, 8 up, 8 in 2024-06-18T13:06:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]': finished 2024-06-18T13:06:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[37701]: osdmap e165: 8 total, 8 up, 8 in 2024-06-18T13:06:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi167-74155-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi167-74155-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:31.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/633379388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi167-73991-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:31.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi167-73991-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:31.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:31.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:31.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:31.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:31.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:31.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:31.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:31.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]': finished 2024-06-18T13:06:31.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:31.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[29496]: osdmap e165: 8 total, 8 up, 8 in 2024-06-18T13:06:31.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi167-74155-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:31.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi167-74155-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:31.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/633379388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi167-73991-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:31.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi167-73991-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:31.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:32.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]': finished 2024-06-18T13:06:32.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:32.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:31 smithi179 ceph-mon[33312]: osdmap e165: 8 total, 8 up, 8 in 2024-06-18T13:06:32.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi167-74155-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:32.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi167-74155-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:32.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/633379388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi167-73991-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:32.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi167-73991-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:32.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:32.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:32 smithi167 ceph-mon[37701]: pgmap v198: 332 pgs: 8 unknown, 324 active+clean; 592 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-06-18T13:06:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi167-73991-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:32 smithi167 ceph-mon[37701]: osdmap e166: 8 total, 8 up, 8 in 2024-06-18T13:06:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:32 smithi167 ceph-mon[29496]: pgmap v198: 332 pgs: 8 unknown, 324 active+clean; 592 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-06-18T13:06:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi167-73991-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:32 smithi167 ceph-mon[29496]: osdmap e166: 8 total, 8 up, 8 in 2024-06-18T13:06:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:33.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:32 smithi179 ceph-mon[33312]: pgmap v198: 332 pgs: 8 unknown, 324 active+clean; 592 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-06-18T13:06:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi167-73991-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:32 smithi179 ceph-mon[33312]: osdmap e166: 8 total, 8 up, 8 in 2024-06-18T13:06:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:34.476 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: Running main() from gmock_main.cc 2024-06-18T13:06:34.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-06-18T13:06:34.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [----------] Global test environment set-up. 2024-06-18T13:06:34.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-06-18T13:06:34.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-06-18T13:06:34.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.TooBig (10355 ms) 2024-06-18T13:06:34.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-06-18T13:06:34.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (7218 ms) 2024-06-18T13:06:34.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-06-18T13:06:34.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (3567 ms) 2024-06-18T13:06:34.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-06-18T13:06:34.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (2836 ms) 2024-06-18T13:06:34.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-06-18T13:06:34.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3078 ms) 2024-06-18T13:06:34.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-06-18T13:06:34.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (5001 ms) 2024-06-18T13:06:34.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-06-18T13:06:34.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3361 ms) 2024-06-18T13:06:34.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-06-18T13:06:34.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (2964 ms) 2024-06-18T13:06:34.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-06-18T13:06:34.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3593 ms) 2024-06-18T13:06:34.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-06-18T13:06:34.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3436 ms) 2024-06-18T13:06:34.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-06-18T13:06:34.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (2938 ms) 2024-06-18T13:06:34.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3014 ms) 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (2993 ms) 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (3250 ms) 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.Flush (2755 ms) 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3001 ms) 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3010 ms) 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3026 ms) 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-06-18T13:06:34.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3004 ms) 2024-06-18T13:06:34.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-06-18T13:06:34.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3011 ms) 2024-06-18T13:06:34.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-06-18T13:06:34.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (2945 ms) 2024-06-18T13:06:34.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-06-18T13:06:34.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3261 ms) 2024-06-18T13:06:34.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-06-18T13:06:34.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (2775 ms) 2024-06-18T13:06:34.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-06-18T13:06:34.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3014 ms) 2024-06-18T13:06:34.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-06-18T13:06:34.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3008 ms) 2024-06-18T13:06:34.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-06-18T13:06:34.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3006 ms) 2024-06-18T13:06:34.481 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [----------] 26 tests from LibRadosAio (93423 ms total) 2024-06-18T13:06:34.481 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: 2024-06-18T13:06:34.481 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-06-18T13:06:34.481 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-06-18T13:06:34.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:34 smithi179 ceph-mon[33312]: pgmap v201: 363 pgs: 40 unknown, 323 active+clean; 592 KiB data, 607 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi167-74155-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi167-74155-36"}]': finished 2024-06-18T13:06:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]': finished 2024-06-18T13:06:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:34 smithi179 ceph-mon[33312]: osdmap e167: 8 total, 8 up, 8 in 2024-06-18T13:06:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:34.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:34 smithi167 ceph-mon[37701]: pgmap v201: 363 pgs: 40 unknown, 323 active+clean; 592 KiB data, 607 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi167-74155-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi167-74155-36"}]': finished 2024-06-18T13:06:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]': finished 2024-06-18T13:06:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:34 smithi167 ceph-mon[37701]: osdmap e167: 8 total, 8 up, 8 in 2024-06-18T13:06:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:34 smithi167 ceph-mon[29496]: pgmap v201: 363 pgs: 40 unknown, 323 active+clean; 592 KiB data, 607 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi167-74155-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi167-74155-36"}]': finished 2024-06-18T13:06:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi167-73976-27"}]': finished 2024-06-18T13:06:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:34 smithi167 ceph-mon[29496]: osdmap e167: 8 total, 8 up, 8 in 2024-06-18T13:06:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3411429039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:34.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]: dispatch 2024-06-18T13:06:34.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:06:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:06:35.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]': finished 2024-06-18T13:06:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: osdmap e168: 8 total, 8 up, 8 in 2024-06-18T13:06:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/249541833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi167-73991-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi167-73991-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi167-73976-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi167-73991-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi167-73976-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:06:35.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: osdmap e169: 8 total, 8 up, 8 in 2024-06-18T13:06:35.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13"}]: dispatch 2024-06-18T13:06:35.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi167-73976-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:35.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:35.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13"}]: dispatch 2024-06-18T13:06:35.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]': finished 2024-06-18T13:06:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: osdmap e168: 8 total, 8 up, 8 in 2024-06-18T13:06:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/249541833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi167-73991-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi167-73991-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi167-73976-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi167-73991-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi167-73976-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:06:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: osdmap e169: 8 total, 8 up, 8 in 2024-06-18T13:06:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13"}]: dispatch 2024-06-18T13:06:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi167-73976-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:35.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:35.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13"}]: dispatch 2024-06-18T13:06:35.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:35.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi167-73976-27"}]': finished 2024-06-18T13:06:35.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: osdmap e168: 8 total, 8 up, 8 in 2024-06-18T13:06:35.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/249541833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi167-73991-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:35.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi167-73991-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:35.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:35.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:35.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi167-73976-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:35.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:35.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:35.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:35.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi167-73991-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:35.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi167-73976-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:35.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:06:35.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: osdmap e169: 8 total, 8 up, 8 in 2024-06-18T13:06:35.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13"}]: dispatch 2024-06-18T13:06:35.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi167-73976-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:35.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:35.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13"}]: dispatch 2024-06-18T13:06:35.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:36 smithi167 ceph-mon[37701]: pgmap v204: 371 pgs: 5 creating+peering, 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 35 unknown, 295 active+clean; 605 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-06-18T13:06:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:36 smithi167 ceph-mon[37701]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:36 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:06:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13"}]': finished 2024-06-18T13:06:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:36 smithi167 ceph-mon[29496]: pgmap v204: 371 pgs: 5 creating+peering, 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 35 unknown, 295 active+clean; 605 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-06-18T13:06:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:36 smithi167 ceph-mon[29496]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:36 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:06:36.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13"}]': finished 2024-06-18T13:06:37.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:36 smithi179 ceph-mon[33312]: pgmap v204: 371 pgs: 5 creating+peering, 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 35 unknown, 295 active+clean; 605 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-06-18T13:06:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:36 smithi179 ceph-mon[33312]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:36 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:06:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-13"}]': finished 2024-06-18T13:06:37.408 INFO:tasks.workunit.client.0.smithi167.stdout:696 ms total) 2024-06-18T13:06:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: 2024-06-18T13:06:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-06-18T13:06:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-06-18T13:06:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (116 ms) 2024-06-18T13:06:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (116 ms total) 2024-06-18T13:06:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: 2024-06-18T13:06:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-06-18T13:06:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-06-18T13:06:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (47 ms) 2024-06-18T13:06:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-06-18T13:06:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (4 ms) 2024-06-18T13:06:37.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (51 ms total) 2024-06-18T13:06:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: 2024-06-18T13:06:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-06-18T13:06:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-06-18T13:06:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (50 ms) 2024-06-18T13:06:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-06-18T13:06:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (4 ms) 2024-06-18T13:06:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (54 ms total) 2024-06-18T13:06:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: 2024-06-18T13:06:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-06-18T13:06:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-06-18T13:06:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (5 ms) 2024-06-18T13:06:37.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-06-18T13:06:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (5 ms) 2024-06-18T13:06:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (10 ms total) 2024-06-18T13:06:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: 2024-06-18T13:06:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-06-18T13:06:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-06-18T13:06:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1042 ms) 2024-06-18T13:06:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1042 ms total) 2024-06-18T13:06:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: 2024-06-18T13:06:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-06-18T13:06:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (103290 ms total) 2024-06-18T13:06:37.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-06-18T13:06:37.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]': finished 2024-06-18T13:06:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[37701]: osdmap e170: 8 total, 8 up, 8 in 2024-06-18T13:06:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi167-73976-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi167-73976-28"}]': finished 2024-06-18T13:06:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]': finished 2024-06-18T13:06:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[37701]: osdmap e171: 8 total, 8 up, 8 in 2024-06-18T13:06:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3998680664' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi167-73991-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi167-73991-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]': finished 2024-06-18T13:06:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[29496]: osdmap e170: 8 total, 8 up, 8 in 2024-06-18T13:06:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi167-73976-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi167-73976-28"}]': finished 2024-06-18T13:06:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]': finished 2024-06-18T13:06:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[29496]: osdmap e171: 8 total, 8 up, 8 in 2024-06-18T13:06:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3998680664' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi167-73991-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi167-73991-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:38.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi167-74155-36"}]': finished 2024-06-18T13:06:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/343239449' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:37 smithi179 ceph-mon[33312]: osdmap e170: 8 total, 8 up, 8 in 2024-06-18T13:06:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]: dispatch 2024-06-18T13:06:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi167-73976-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi167-73976-28"}]': finished 2024-06-18T13:06:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi167-74155-36"}]': finished 2024-06-18T13:06:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:37 smithi179 ceph-mon[33312]: osdmap e171: 8 total, 8 up, 8 in 2024-06-18T13:06:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3998680664' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi167-73991-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi167-73991-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:38.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:06:38 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:06:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:38 smithi179 ceph-mon[33312]: pgmap v208: 339 pgs: 40 unknown, 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 280 active+clean; 603 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s wr, 2 op/s; 32 B/s, 0 objects/s recovering 2024-06-18T13:06:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:38 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:38 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:06:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:38 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:06:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:38 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:06:38.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:38 smithi179 ceph-mon[33312]: pgmap v209: 339 pgs: 40 unknown, 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 280 active+clean; 603 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:38.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:38 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:06:38.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:38 smithi179 ceph-mon[33312]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-06-18T13:06:38.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi167-73991-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:38.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:38 smithi179 ceph-mon[33312]: osdmap e172: 8 total, 8 up, 8 in 2024-06-18T13:06:38.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:38 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:38.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:38.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[37701]: pgmap v208: 339 pgs: 40 unknown, 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 280 active+clean; 603 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s wr, 2 op/s; 32 B/s, 0 objects/s recovering 2024-06-18T13:06:38.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:38.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:06:38.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:06:38.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:06:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[37701]: pgmap v209: 339 pgs: 40 unknown, 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 280 active+clean; 603 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:06:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[37701]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-06-18T13:06:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi167-73991-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[37701]: osdmap e172: 8 total, 8 up, 8 in 2024-06-18T13:06:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:38.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[29496]: pgmap v208: 339 pgs: 40 unknown, 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 280 active+clean; 603 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s wr, 2 op/s; 32 B/s, 0 objects/s recovering 2024-06-18T13:06:38.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:38.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:06:38.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:06:38.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:06:38.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[29496]: pgmap v209: 339 pgs: 40 unknown, 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 280 active+clean; 603 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:38.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:06:38.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[29496]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-06-18T13:06:38.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi167-73991-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:38.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[29496]: osdmap e172: 8 total, 8 up, 8 in 2024-06-18T13:06:38.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:38.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:40.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:39 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:40.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:40.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:06:40.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:06:40.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:39 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:40.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:39 smithi179 ceph-mon[33312]: osdmap e173: 8 total, 8 up, 8 in 2024-06-18T13:06:40.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:40.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:39 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:40.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:40.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:06:40.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:06:40.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:39 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:40.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:39 smithi167 ceph-mon[37701]: osdmap e173: 8 total, 8 up, 8 in 2024-06-18T13:06:40.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:40.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:39 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:40.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:40.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:06:40.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:06:40.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:39 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:40.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:39 smithi167 ceph-mon[29496]: osdmap e173: 8 total, 8 up, 8 in 2024-06-18T13:06:40.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:41.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:40 smithi179 ceph-mon[33312]: pgmap v212: 331 pgs: 32 creating+peering, 299 active+clean; 610 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 8.8 KiB/s wr, 1 op/s 2024-06-18T13:06:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi167-73976-28"}]': finished 2024-06-18T13:06:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:40 smithi179 ceph-mon[33312]: osdmap e174: 8 total, 8 up, 8 in 2024-06-18T13:06:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3001814148' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi167-73991-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi167-73991-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:41.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:41.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[29496]: pgmap v212: 331 pgs: 32 creating+peering, 299 active+clean; 610 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 8.8 KiB/s wr, 1 op/s 2024-06-18T13:06:41.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi167-73976-28"}]': finished 2024-06-18T13:06:41.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[29496]: osdmap e174: 8 total, 8 up, 8 in 2024-06-18T13:06:41.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3001814148' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi167-73991-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:41.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:41.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi167-73991-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:41.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:41.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:41.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:41.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[37701]: pgmap v212: 331 pgs: 32 creating+peering, 299 active+clean; 610 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 8.8 KiB/s wr, 1 op/s 2024-06-18T13:06:41.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi167-73976-28"}]': finished 2024-06-18T13:06:41.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[37701]: osdmap e174: 8 total, 8 up, 8 in 2024-06-18T13:06:41.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3001814148' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi167-73991-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:41.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi167-73976-28"}]: dispatch 2024-06-18T13:06:41.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi167-73991-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:41.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:41.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:41 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi167-73976-28"}]': finished 2024-06-18T13:06:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi167-73991-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:06:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:41 smithi179 ceph-mon[33312]: osdmap e175: 8 total, 8 up, 8 in 2024-06-18T13:06:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-15"}]: dispatch 2024-06-18T13:06:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-15"}]: dispatch 2024-06-18T13:06:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:42.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:42.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi167-73976-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:42.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:42.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:42.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi167-73976-28"}]': finished 2024-06-18T13:06:42.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi167-73991-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:42.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:06:42.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[29496]: osdmap e175: 8 total, 8 up, 8 in 2024-06-18T13:06:42.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-15"}]: dispatch 2024-06-18T13:06:42.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-15"}]: dispatch 2024-06-18T13:06:42.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:42.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:42.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi167-73976-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:42.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:42.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:42.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4163072380' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi167-73976-28"}]': finished 2024-06-18T13:06:42.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi167-73991-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:42.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:06:42.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[37701]: osdmap e175: 8 total, 8 up, 8 in 2024-06-18T13:06:42.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-15"}]: dispatch 2024-06-18T13:06:42.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-15"}]: dispatch 2024-06-18T13:06:42.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:42.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:42.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi167-73976-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:43.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:42 smithi179 ceph-mon[33312]: pgmap v215: 363 pgs: 32 unknown, 32 creating+peering, 299 active+clean; 610 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 9.2 KiB/s rd, 8.0 KiB/s wr, 4 op/s 2024-06-18T13:06:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-15"}]': finished 2024-06-18T13:06:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi167-73976-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:42 smithi179 ceph-mon[33312]: osdmap e176: 8 total, 8 up, 8 in 2024-06-18T13:06:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi167-73976-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-15", "mode": "writeback"}]: dispatch 2024-06-18T13:06:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-15", "mode": "writeback"}]: dispatch 2024-06-18T13:06:43.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:43.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[29496]: pgmap v215: 363 pgs: 32 unknown, 32 creating+peering, 299 active+clean; 610 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 9.2 KiB/s rd, 8.0 KiB/s wr, 4 op/s 2024-06-18T13:06:43.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-15"}]': finished 2024-06-18T13:06:43.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi167-73976-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:43.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[29496]: osdmap e176: 8 total, 8 up, 8 in 2024-06-18T13:06:43.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi167-73976-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:43.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-15", "mode": "writeback"}]: dispatch 2024-06-18T13:06:43.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-15", "mode": "writeback"}]: dispatch 2024-06-18T13:06:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[37701]: pgmap v215: 363 pgs: 32 unknown, 32 creating+peering, 299 active+clean; 610 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 9.2 KiB/s rd, 8.0 KiB/s wr, 4 op/s 2024-06-18T13:06:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-15"}]': finished 2024-06-18T13:06:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi167-73976-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[37701]: osdmap e176: 8 total, 8 up, 8 in 2024-06-18T13:06:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi167-73976-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-15", "mode": "writeback"}]: dispatch 2024-06-18T13:06:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-15", "mode": "writeback"}]: dispatch 2024-06-18T13:06:44.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:43 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:06:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-15", "mode": "writeback"}]': finished 2024-06-18T13:06:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:43 smithi179 ceph-mon[33312]: osdmap e177: 8 total, 8 up, 8 in 2024-06-18T13:06:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4261826446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi167-73991-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:44.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:44.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:43 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:06:44.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-15", "mode": "writeback"}]': finished 2024-06-18T13:06:44.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:43 smithi167 ceph-mon[37701]: osdmap e177: 8 total, 8 up, 8 in 2024-06-18T13:06:44.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4261826446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi167-73991-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:44.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:44.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:44.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:43 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:06:44.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-15", "mode": "writeback"}]': finished 2024-06-18T13:06:44.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:43 smithi167 ceph-mon[29496]: osdmap e177: 8 total, 8 up, 8 in 2024-06-18T13:06:44.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4261826446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi167-73991-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:44.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:45.407 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:06:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:06:45.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-06-18T13:06:45.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-06-18T13:06:45.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-06-18T13:06:45.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-06-18T13:06:45.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-06-18T13:06:45.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (10056 ms) 2024-06-18T13:06:45.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-06-18T13:06:45.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (20394 ms) 2024-06-18T13:06:45.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-06-18T13:06:45.408 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (7921 ms) 2024-06-18T13:06:45.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-06-18T13:06:45.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (3616 ms) 2024-06-18T13:06:45.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-06-18T13:06:45.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3414 ms) 2024-06-18T13:06:45.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-06-18T13:06:45.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (2944 ms) 2024-06-18T13:06:45.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-06-18T13:06:45.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3016 ms) 2024-06-18T13:06:45.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-06-18T13:06:45.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (2987 ms) 2024-06-18T13:06:45.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-06-18T13:06:45.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (3256 ms) 2024-06-18T13:06:45.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-06-18T13:06:45.409 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (2766 ms) 2024-06-18T13:06:45.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-06-18T13:06:45.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2979 ms) 2024-06-18T13:06:45.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-06-18T13:06:45.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3020 ms) 2024-06-18T13:06:45.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-06-18T13:06:45.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3015 ms) 2024-06-18T13:06:45.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-06-18T13:06:45.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3013 ms) 2024-06-18T13:06:45.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-06-18T13:06:45.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3016 ms) 2024-06-18T13:06:45.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-06-18T13:06:45.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (2953 ms) 2024-06-18T13:06:45.410 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3246 ms) 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (2779 ms) 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3013 ms) 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3007 ms) 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3007 ms) 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3251 ms) 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (2769 ms) 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-06-18T13:06:45.411 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3248 ms) 2024-06-18T13:06:45.412 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-06-18T13:06:45.412 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2684 ms) 2024-06-18T13:06:45.412 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-06-18T13:06:45.412 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (2991 ms) 2024-06-18T13:06:45.412 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-06-18T13:06:45.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[37701]: pgmap v218: 363 pgs: 20 creating+peering, 12 unknown, 331 active+clean; 610 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-06-18T13:06:45.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi167-73976-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi167-73976-29"}]': finished 2024-06-18T13:06:45.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4261826446' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi167-73991-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:45.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[37701]: osdmap e178: 8 total, 8 up, 8 in 2024-06-18T13:06:45.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:45.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:45.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:45.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:45.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[29496]: pgmap v218: 363 pgs: 20 creating+peering, 12 unknown, 331 active+clean; 610 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-06-18T13:06:45.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi167-73976-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi167-73976-29"}]': finished 2024-06-18T13:06:45.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4261826446' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi167-73991-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:45.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[29496]: osdmap e178: 8 total, 8 up, 8 in 2024-06-18T13:06:45.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:45.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:45.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:45.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:45 smithi179 ceph-mon[33312]: pgmap v218: 363 pgs: 20 creating+peering, 12 unknown, 331 active+clean; 610 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-06-18T13:06:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi167-73976-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi167-73976-29"}]': finished 2024-06-18T13:06:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4261826446' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi167-73991-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:45 smithi179 ceph-mon[33312]: osdmap e178: 8 total, 8 up, 8 in 2024-06-18T13:06:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:45 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:46.420 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAihotsECPP.SnapRemovePP (1995 ms) 2024-06-18T13:06:46.420 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-06-18T13:06:46.420 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2011 ms) 2024-06-18T13:06:46.420 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-06-18T13:06:46.420 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2002 ms) 2024-06-18T13:06:46.420 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (9033 ms total) 2024-06-18T13:06:46.420 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: 2024-06-18T13:06:46.420 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-06-18T13:06:46.421 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-06-18T13:06:46.421 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4018 ms) 2024-06-18T13:06:46.421 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-06-18T13:06:46.421 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3914 ms) 2024-06-18T13:06:46.421 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-06-18T13:06:46.421 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4235 ms) 2024-06-18T13:06:46.421 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (12167 ms total) 2024-06-18T13:06:46.421 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: 2024-06-18T13:06:46.421 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-06-18T13:06:46.421 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (112249 ms total) 2024-06-18T13:06:46.421 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-06-18T13:06:46.421 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-06-18T13:06:46.422 INFO:tasks.workunit.client.0.smithi167.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-06-18T13:06:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]': finished 2024-06-18T13:06:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:06:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[37701]: osdmap e179: 8 total, 8 up, 8 in 2024-06-18T13:06:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15"}]: dispatch 2024-06-18T13:06:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15"}]: dispatch 2024-06-18T13:06:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[37701]: pgmap v221: 331 pgs: 8 unknown, 323 active+clean; 586 KiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:06:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]': finished 2024-06-18T13:06:46.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:06:46.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[29496]: osdmap e179: 8 total, 8 up, 8 in 2024-06-18T13:06:46.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15"}]: dispatch 2024-06-18T13:06:46.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:46.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15"}]: dispatch 2024-06-18T13:06:46.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:46.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[29496]: pgmap v221: 331 pgs: 8 unknown, 323 active+clean; 586 KiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:46.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:46 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:06:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]': finished 2024-06-18T13:06:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:06:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:46 smithi179 ceph-mon[33312]: osdmap e179: 8 total, 8 up, 8 in 2024-06-18T13:06:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15"}]: dispatch 2024-06-18T13:06:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]: dispatch 2024-06-18T13:06:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15"}]: dispatch 2024-06-18T13:06:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:46 smithi179 ceph-mon[33312]: pgmap v221: 331 pgs: 8 unknown, 323 active+clean; 586 KiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:46 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:06:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]': finished 2024-06-18T13:06:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15"}]': finished 2024-06-18T13:06:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[37701]: osdmap e180: 8 total, 8 up, 8 in 2024-06-18T13:06:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/292108443' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi167-73991-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi167-73991-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi167-73991-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi167-73976-29"}]': finished 2024-06-18T13:06:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[37701]: osdmap e181: 8 total, 8 up, 8 in 2024-06-18T13:06:47.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]': finished 2024-06-18T13:06:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15"}]': finished 2024-06-18T13:06:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[29496]: osdmap e180: 8 total, 8 up, 8 in 2024-06-18T13:06:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/292108443' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi167-73991-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi167-73991-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi167-73991-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi167-73976-29"}]': finished 2024-06-18T13:06:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[29496]: osdmap e181: 8 total, 8 up, 8 in 2024-06-18T13:06:47.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:48.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2804196285' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi167-74374-21"}]': finished 2024-06-18T13:06:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-15"}]': finished 2024-06-18T13:06:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:47 smithi179 ceph-mon[33312]: osdmap e180: 8 total, 8 up, 8 in 2024-06-18T13:06:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/292108443' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi167-73991-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:47 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi167-73991-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi167-73991-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi167-73976-29"}]': finished 2024-06-18T13:06:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:47 smithi179 ceph-mon[33312]: osdmap e181: 8 total, 8 up, 8 in 2024-06-18T13:06:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi167-73976-29"}]: dispatch 2024-06-18T13:06:48.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:48.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:48 smithi167 ceph-mon[37701]: pgmap v224: 323 pgs: 32 unknown, 291 active+clean; 586 KiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi167-73976-29"}]': finished 2024-06-18T13:06:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:48 smithi167 ceph-mon[37701]: osdmap e182: 8 total, 8 up, 8 in 2024-06-18T13:06:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:48 smithi167 ceph-mon[29496]: pgmap v224: 323 pgs: 32 unknown, 291 active+clean; 586 KiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi167-73976-29"}]': finished 2024-06-18T13:06:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:48 smithi167 ceph-mon[29496]: osdmap e182: 8 total, 8 up, 8 in 2024-06-18T13:06:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:49.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:06:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:06:49.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:49.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:48 smithi179 ceph-mon[33312]: pgmap v224: 323 pgs: 32 unknown, 291 active+clean; 586 KiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:49.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3627493418' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi167-73976-29"}]': finished 2024-06-18T13:06:49.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:48 smithi179 ceph-mon[33312]: osdmap e182: 8 total, 8 up, 8 in 2024-06-18T13:06:49.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:49.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi167-73976-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi167-73976-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[29496]: osdmap e183: 8 total, 8 up, 8 in 2024-06-18T13:06:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi167-73976-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/364013988' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi167-73991-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi167-73991-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:49.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:49.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:49.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:49.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:49.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:49.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi167-73976-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:49.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:49.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:49.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi167-73976-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:49.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[37701]: osdmap e183: 8 total, 8 up, 8 in 2024-06-18T13:06:49.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi167-73976-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:49.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/364013988' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi167-73991-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:49.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi167-73991-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:49.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:49.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:50.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:49 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi167-73976-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi167-73976-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:49 smithi179 ceph-mon[33312]: osdmap e183: 8 total, 8 up, 8 in 2024-06-18T13:06:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi167-73976-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/364013988' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi167-73991-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi167-73991-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:50.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:50.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:50.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:50 smithi167 ceph-mon[37701]: pgmap v227: 355 pgs: 32 unknown, 32 creating+peering, 291 active+clean; 586 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:06:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi167-73991-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:06:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-17"}]: dispatch 2024-06-18T13:06:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:50 smithi167 ceph-mon[37701]: osdmap e184: 8 total, 8 up, 8 in 2024-06-18T13:06:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-17"}]: dispatch 2024-06-18T13:06:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:50 smithi167 ceph-mon[29496]: pgmap v227: 355 pgs: 32 unknown, 32 creating+peering, 291 active+clean; 586 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:06:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi167-73991-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:50.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:06:50.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-17"}]: dispatch 2024-06-18T13:06:50.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:50 smithi167 ceph-mon[29496]: osdmap e184: 8 total, 8 up, 8 in 2024-06-18T13:06:50.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-17"}]: dispatch 2024-06-18T13:06:51.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:50 smithi179 ceph-mon[33312]: pgmap v227: 355 pgs: 32 unknown, 32 creating+peering, 291 active+clean; 586 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:06:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi167-73991-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:06:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-17"}]: dispatch 2024-06-18T13:06:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:50 smithi179 ceph-mon[33312]: osdmap e184: 8 total, 8 up, 8 in 2024-06-18T13:06:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-17"}]: dispatch 2024-06-18T13:06:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:51 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:51 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi167-73976-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi167-73976-30"}]': finished 2024-06-18T13:06:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:51 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-17"}]': finished 2024-06-18T13:06:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:51 smithi167 ceph-mon[37701]: osdmap e185: 8 total, 8 up, 8 in 2024-06-18T13:06:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:51 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-17", "mode": "writeback"}]: dispatch 2024-06-18T13:06:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:51 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-17", "mode": "writeback"}]: dispatch 2024-06-18T13:06:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:51 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:51 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi167-73976-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi167-73976-30"}]': finished 2024-06-18T13:06:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:51 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-17"}]': finished 2024-06-18T13:06:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:51 smithi167 ceph-mon[29496]: osdmap e185: 8 total, 8 up, 8 in 2024-06-18T13:06:51.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:51 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-17", "mode": "writeback"}]: dispatch 2024-06-18T13:06:51.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:51 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-17", "mode": "writeback"}]: dispatch 2024-06-18T13:06:52.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:51 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:51 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi167-73976-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi167-73976-30"}]': finished 2024-06-18T13:06:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:51 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-17"}]': finished 2024-06-18T13:06:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:51 smithi179 ceph-mon[33312]: osdmap e185: 8 total, 8 up, 8 in 2024-06-18T13:06:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:51 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-17", "mode": "writeback"}]: dispatch 2024-06-18T13:06:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:51 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-17", "mode": "writeback"}]: dispatch 2024-06-18T13:06:52.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[37701]: pgmap v230: 331 pgs: 8 unknown, 32 creating+peering, 291 active+clean; 586 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:06:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:06:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-17", "mode": "writeback"}]': finished 2024-06-18T13:06:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[37701]: osdmap e186: 8 total, 8 up, 8 in 2024-06-18T13:06:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/934008038' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi167-73991-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:52.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[29496]: pgmap v230: 331 pgs: 8 unknown, 32 creating+peering, 291 active+clean; 586 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:06:52.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:06:52.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-17", "mode": "writeback"}]': finished 2024-06-18T13:06:52.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[29496]: osdmap e186: 8 total, 8 up, 8 in 2024-06-18T13:06:52.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/934008038' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi167-73991-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:52.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:52.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:52 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:53.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:52 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:53.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:52 smithi179 ceph-mon[33312]: pgmap v230: 331 pgs: 8 unknown, 32 creating+peering, 291 active+clean; 586 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:06:53.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:52 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:06:53.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:52 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-17", "mode": "writeback"}]': finished 2024-06-18T13:06:53.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:52 smithi179 ceph-mon[33312]: osdmap e186: 8 total, 8 up, 8 in 2024-06-18T13:06:53.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:52 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/934008038' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi167-73991-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:53.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:52 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:53.039 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:52 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:06:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/934008038' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi167-73991-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:06:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:53 smithi167 ceph-mon[37701]: osdmap e187: 8 total, 8 up, 8 in 2024-06-18T13:06:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17"}]: dispatch 2024-06-18T13:06:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17"}]: dispatch 2024-06-18T13:06:53.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:53.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/934008038' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi167-73991-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:53.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:06:53.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:53 smithi167 ceph-mon[29496]: osdmap e187: 8 total, 8 up, 8 in 2024-06-18T13:06:53.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:53.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17"}]: dispatch 2024-06-18T13:06:53.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17"}]: dispatch 2024-06-18T13:06:54.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/934008038' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi167-73991-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:06:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:53 smithi179 ceph-mon[33312]: osdmap e187: 8 total, 8 up, 8 in 2024-06-18T13:06:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17"}]: dispatch 2024-06-18T13:06:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17"}]: dispatch 2024-06-18T13:06:54.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[37701]: pgmap v233: 355 pgs: 355 active+clean; 586 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-06-18T13:06:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-06-18T13:06:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:06:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:06:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:06:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi167-73976-30"}]': finished 2024-06-18T13:06:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17"}]': finished 2024-06-18T13:06:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-06-18T13:06:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[37701]: osdmap e188: 8 total, 8 up, 8 in 2024-06-18T13:06:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[29496]: pgmap v233: 355 pgs: 355 active+clean; 586 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-06-18T13:06:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-06-18T13:06:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:06:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:06:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:06:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi167-73976-30"}]': finished 2024-06-18T13:06:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17"}]': finished 2024-06-18T13:06:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-06-18T13:06:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[29496]: osdmap e188: 8 total, 8 up, 8 in 2024-06-18T13:06:54.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:54 smithi179 ceph-mon[33312]: pgmap v233: 355 pgs: 355 active+clean; 586 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-06-18T13:06:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:54 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-06-18T13:06:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:54 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:06:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:54 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:06:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:54 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:06:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi167-73976-30"}]': finished 2024-06-18T13:06:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-17"}]': finished 2024-06-18T13:06:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:54 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi167-74075-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-06-18T13:06:55.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:54 smithi179 ceph-mon[33312]: osdmap e188: 8 total, 8 up, 8 in 2024-06-18T13:06:55.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi167-73976-30"}]: dispatch 2024-06-18T13:06:55.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:06:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:06:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]: dispatch 2024-06-18T13:06:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]: dispatch 2024-06-18T13:06:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi167-73976-30"}]': finished 2024-06-18T13:06:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]': finished 2024-06-18T13:06:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: osdmap e189: 8 total, 8 up, 8 in 2024-06-18T13:06:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-18T13:06:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3180330776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi167-73991-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi167-73991-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:55.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-18T13:06:55.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:06:55.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:06:55.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi167-73976-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]: dispatch 2024-06-18T13:06:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]: dispatch 2024-06-18T13:06:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi167-73976-30"}]': finished 2024-06-18T13:06:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]': finished 2024-06-18T13:06:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: osdmap e189: 8 total, 8 up, 8 in 2024-06-18T13:06:55.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:55.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:55.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-18T13:06:55.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3180330776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi167-73991-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:55.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi167-73991-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:55.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-18T13:06:55.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:06:55.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:06:55.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi167-73976-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:06:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]: dispatch 2024-06-18T13:06:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]: dispatch 2024-06-18T13:06:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2724198432' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi167-73976-30"}]': finished 2024-06-18T13:06:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pg_num","val":"11"}]': finished 2024-06-18T13:06:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: osdmap e189: 8 total, 8 up, 8 in 2024-06-18T13:06:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-18T13:06:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3180330776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi167-73991-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:56.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi167-73991-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:56.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-18T13:06:56.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:06:56.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:06:56.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi167-73976-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:56.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[37701]: pgmap v236: 323 pgs: 32 unknown, 291 active+clean; 586 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:06:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi167-73991-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]': finished 2024-06-18T13:06:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi167-73976-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[37701]: osdmap e190: 8 total, 8 up, 8 in 2024-06-18T13:06:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi167-73976-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:06:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[29496]: pgmap v236: 323 pgs: 32 unknown, 291 active+clean; 586 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:06:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi167-73991-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]': finished 2024-06-18T13:06:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi167-73976-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[29496]: osdmap e190: 8 total, 8 up, 8 in 2024-06-18T13:06:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi167-73976-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:06:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:57.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:56 smithi179 ceph-mon[33312]: pgmap v236: 323 pgs: 32 unknown, 291 active+clean; 586 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:06:57.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi167-73991-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi167-74075-1","var":"pgp_num","val":"11"}]': finished 2024-06-18T13:06:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi167-73976-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:56 smithi179 ceph-mon[33312]: osdmap e190: 8 total, 8 up, 8 in 2024-06-18T13:06:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3722472229' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-18T13:06:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi167-73976-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:06:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3180330776' entity='client.admin' cmd=[{ 2024-06-18T13:06:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: "prefix": "osd pool set", 2024-06-18T13:06:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: "pool": "PoolEIOFlag_smithi167-73991-33", 2024-06-18T13:06:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: "var": "eio", 2024-06-18T13:06:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: "val": "true" 2024-06-18T13:06:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: }]: dispatch 2024-06-18T13:06:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{ 2024-06-18T13:06:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: "prefix": "osd pool set", 2024-06-18T13:06:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: "pool": "PoolEIOFlag_smithi167-73991-33", 2024-06-18T13:06:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: "var": "eio", 2024-06-18T13:06:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: "val": "true" 2024-06-18T13:06:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: }]: dispatch 2024-06-18T13:06:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{ 2024-06-18T13:06:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: "prefix": "osd pool set", 2024-06-18T13:06:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: "pool": "PoolEIOFlag_smithi167-73991-33", 2024-06-18T13:06:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: "var": "eio", 2024-06-18T13:06:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: "val": "true" 2024-06-18T13:06:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: }]': finished 2024-06-18T13:06:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: osdmap e191: 8 total, 8 up, 8 in 2024-06-18T13:06:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:06:57.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:06:57.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi167-74075-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:57.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3180330776' entity='client.admin' cmd=[{ 2024-06-18T13:06:57.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: "prefix": "osd pool set", 2024-06-18T13:06:57.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: "pool": "PoolEIOFlag_smithi167-73991-33", 2024-06-18T13:06:57.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: "var": "eio", 2024-06-18T13:06:57.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: "val": "true" 2024-06-18T13:06:57.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: }]: dispatch 2024-06-18T13:06:57.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{ 2024-06-18T13:06:57.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: "prefix": "osd pool set", 2024-06-18T13:06:57.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: "pool": "PoolEIOFlag_smithi167-73991-33", 2024-06-18T13:06:57.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: "var": "eio", 2024-06-18T13:06:57.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: "val": "true" 2024-06-18T13:06:57.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: }]: dispatch 2024-06-18T13:06:57.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:57.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{ 2024-06-18T13:06:57.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: "prefix": "osd pool set", 2024-06-18T13:06:57.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: "pool": "PoolEIOFlag_smithi167-73991-33", 2024-06-18T13:06:57.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: "var": "eio", 2024-06-18T13:06:57.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: "val": "true" 2024-06-18T13:06:57.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: }]': finished 2024-06-18T13:06:57.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:57.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: osdmap e191: 8 total, 8 up, 8 in 2024-06-18T13:06:57.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:57.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:06:57.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:06:57.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi167-74075-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3180330776' entity='client.admin' cmd=[{ 2024-06-18T13:06:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: "prefix": "osd pool set", 2024-06-18T13:06:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: "pool": "PoolEIOFlag_smithi167-73991-33", 2024-06-18T13:06:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: "var": "eio", 2024-06-18T13:06:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: "val": "true" 2024-06-18T13:06:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: }]: dispatch 2024-06-18T13:06:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{ 2024-06-18T13:06:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: "prefix": "osd pool set", 2024-06-18T13:06:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: "pool": "PoolEIOFlag_smithi167-73991-33", 2024-06-18T13:06:58.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: "var": "eio", 2024-06-18T13:06:58.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: "val": "true" 2024-06-18T13:06:58.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: }]: dispatch 2024-06-18T13:06:58.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:06:58.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{ 2024-06-18T13:06:58.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: "prefix": "osd pool set", 2024-06-18T13:06:58.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: "pool": "PoolEIOFlag_smithi167-73991-33", 2024-06-18T13:06:58.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: "var": "eio", 2024-06-18T13:06:58.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: "val": "true" 2024-06-18T13:06:58.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: }]': finished 2024-06-18T13:06:58.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:58.139 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: osdmap e191: 8 total, 8 up, 8 in 2024-06-18T13:06:58.139 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:06:58.139 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:06:58.139 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:06:58.139 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi167-74075-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:06:58.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[37701]: pgmap v239: 324 pgs: 64 unknown, 260 active+clean; 584 KiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi167-73976-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi167-73976-31"}]': finished 2024-06-18T13:06:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:06:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi167-74075-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[37701]: osdmap e192: 8 total, 8 up, 8 in 2024-06-18T13:06:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-19"}]: dispatch 2024-06-18T13:06:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi167-74075-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:06:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-19"}]: dispatch 2024-06-18T13:06:58.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[29496]: pgmap v239: 324 pgs: 64 unknown, 260 active+clean; 584 KiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:58.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi167-73976-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi167-73976-31"}]': finished 2024-06-18T13:06:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:06:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi167-74075-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[29496]: osdmap e192: 8 total, 8 up, 8 in 2024-06-18T13:06:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-19"}]: dispatch 2024-06-18T13:06:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi167-74075-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:06:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-19"}]: dispatch 2024-06-18T13:06:59.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:06:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:06:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:58 smithi179 ceph-mon[33312]: pgmap v239: 324 pgs: 64 unknown, 260 active+clean; 584 KiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:06:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi167-73976-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi167-73976-31"}]': finished 2024-06-18T13:06:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:06:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi167-74075-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:06:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:58 smithi179 ceph-mon[33312]: osdmap e192: 8 total, 8 up, 8 in 2024-06-18T13:06:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-19"}]: dispatch 2024-06-18T13:06:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi167-74075-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:06:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-19"}]: dispatch 2024-06-18T13:06:59.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:59 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-19"}]': finished 2024-06-18T13:06:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:59 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-19", "mode": "writeback"}]: dispatch 2024-06-18T13:06:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:59 smithi167 ceph-mon[37701]: osdmap e193: 8 total, 8 up, 8 in 2024-06-18T13:06:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-19", "mode": "writeback"}]: dispatch 2024-06-18T13:06:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:59 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1669780862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi167-73991-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:06:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi167-73991-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:59 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:06:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-19"}]': finished 2024-06-18T13:06:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:59 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-19", "mode": "writeback"}]: dispatch 2024-06-18T13:06:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:59 smithi167 ceph-mon[29496]: osdmap e193: 8 total, 8 up, 8 in 2024-06-18T13:06:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-19", "mode": "writeback"}]: dispatch 2024-06-18T13:06:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:59 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1669780862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi167-73991-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:06:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:06:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi167-73991-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:00.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:59 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-19"}]': finished 2024-06-18T13:07:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:59 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-19", "mode": "writeback"}]: dispatch 2024-06-18T13:07:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:59 smithi179 ceph-mon[33312]: osdmap e193: 8 total, 8 up, 8 in 2024-06-18T13:07:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-19", "mode": "writeback"}]: dispatch 2024-06-18T13:07:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:59 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1669780862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi167-73991-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:06:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi167-73991-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:01.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:00 smithi179 ceph-mon[33312]: pgmap v242: 332 pgs: 32 unknown, 8 creating+peering, 292 active+clean; 584 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:00 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi167-74075-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi167-74075-2"}]': finished 2024-06-18T13:07:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-19", "mode": "writeback"}]': finished 2024-06-18T13:07:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi167-73991-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:00 smithi179 ceph-mon[33312]: osdmap e194: 8 total, 8 up, 8 in 2024-06-18T13:07:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:07:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[29496]: pgmap v242: 332 pgs: 32 unknown, 8 creating+peering, 292 active+clean; 584 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi167-74075-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi167-74075-2"}]': finished 2024-06-18T13:07:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-19", "mode": "writeback"}]': finished 2024-06-18T13:07:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi167-73991-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[29496]: osdmap e194: 8 total, 8 up, 8 in 2024-06-18T13:07:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:07:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[37701]: pgmap v242: 332 pgs: 32 unknown, 8 creating+peering, 292 active+clean; 584 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:01.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:01.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi167-74075-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi167-74075-2"}]': finished 2024-06-18T13:07:01.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-19", "mode": "writeback"}]': finished 2024-06-18T13:07:01.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi167-73991-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:01.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[37701]: osdmap e194: 8 total, 8 up, 8 in 2024-06-18T13:07:01.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:07:01.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:01.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:01.601 INFO:tasks.workunit.client.0.smithi167.stdout:expected=3 2024-06-18T13:07:01.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:bd63b0f1:::8:head 2024-06-18T13:07:01.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:bd63b0f1:::8:head expected=21:bd63b0f1:::8:head 2024-06-18T13:07:01.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:bd63b0f1:::8:head -> 8 2024-06-18T13:07:01.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=8 expected=8 2024-06-18T13:07:01.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:b29083e3:::5:head 2024-06-18T13:07:01.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:b29083e3:::5:head expected=21:b29083e3:::5:head 2024-06-18T13:07:01.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:b29083e3:::5:head -> 5 2024-06-18T13:07:01.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=5 expected=5 2024-06-18T13:07:01.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:89d3ae78:::11:head 2024-06-18T13:07:01.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:89d3ae78:::11:head expected=21:89d3ae78:::11:head 2024-06-18T13:07:01.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:89d3ae78:::11:head -> 11 2024-06-18T13:07:01.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=11 expected=11 2024-06-18T13:07:01.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:863748b0:::15:head 2024-06-18T13:07:01.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:863748b0:::15:head expected=21:863748b0:::15:head 2024-06-18T13:07:01.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:863748b0:::15:head -> 15 2024-06-18T13:07:01.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=15 expected=15 2024-06-18T13:07:01.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:62a1935d:::14:head 2024-06-18T13:07:01.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:62a1935d:::14:head expected=21:62a1935d:::14:head 2024-06-18T13:07:01.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:62a1935d:::14:head -> 14 2024-06-18T13:07:01.603 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=14 expected=14 2024-06-18T13:07:01.603 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:5c6b0b28:::7:head 2024-06-18T13:07:01.603 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:5c6b0b28:::7:head expected=21:5c6b0b28:::7:head 2024-06-18T13:07:01.603 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:5c6b0b28:::7:head -> 7 2024-06-18T13:07:01.603 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=7 expected=7 2024-06-18T13:07:01.603 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:02547ec2:::1:head 2024-06-18T13:07:01.603 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:02547ec2:::1:head expected=21:02547ec2:::1:head 2024-06-18T13:07:01.603 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:02547ec2:::1:head -> 1 2024-06-18T13:07:01.603 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=1 expected=1 2024-06-18T13:07:01.603 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : seek to 21:566253c9:::13:head 2024-06-18T13:07:01.603 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : cursor()=21:566253c9:::13:head expected=21:566253c9:::13:head 2024-06-18T13:07:01.603 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: > 21:566253c9:::13:head -> 13 2024-06-18T13:07:01.603 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: : entry=13 expected=13 2024-06-18T13:07:01.603 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (243 ms) 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (80161 ms) 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: split 0/5 -> MIN 21:33333333::::head 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: split 1/5 -> 21:33333333::::head 21:66666666::::head 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: split 2/5 -> 21:66666666::::head 21:99999999::::head 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: split 3/5 -> 21:99999999::::head 21:cccccccc::::head 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: split 4/5 -> 21:cccccccc::::head MAX 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (31775 ms) 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [----------] 7 tests from LibRadosList (112706 ms total) 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1152 ms) 2024-06-18T13:07:01.604 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-06-18T13:07:01.605 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: myset foo1,foo2,foo3 2024-06-18T13:07:01.605 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo1 2024-06-18T13:07:01.605 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo2 2024-06-18T13:07:01.605 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo3 2024-06-18T13:07:01.605 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: myset foo1,foo4,foo5 2024-06-18T13:07:01.605 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo4 2024-06-18T13:07:01.605 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo5 2024-06-18T13:07:01.605 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo1 2024-06-18T13:07:01.605 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: myset foo6,foo7 2024-06-18T13:07:01.605 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo7 2024-06-18T13:07:01.605 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: foo6 2024-06-18T13:07:01.605 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-06-18T13:07:01.605 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: ns1:foo4 2024-06-18T13:07:01.605 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: ns1:foo5 2024-06-18T13:07:01.606 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: ns2:foo7 2024-06-18T13:07:01.606 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: ns2:foo6 2024-06-18T13:07:01.606 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: ns1:foo1 2024-06-18T13:07:01.606 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: :foo1 2024-06-18T13:07:01.606 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: :foo2 2024-06-18T13:07:01.606 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: :foo3 2024-06-18T13:07:01.606 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (29 ms) 2024-06-18T13:07:02.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:01 smithi179 ceph-mon[33312]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:02.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi167-73976-31"}]': finished 2024-06-18T13:07:02.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:02.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:01 smithi179 ceph-mon[33312]: osdmap e195: 8 total, 8 up, 8 in 2024-06-18T13:07:02.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:07:02.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19"}]: dispatch 2024-06-18T13:07:02.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19"}]: dispatch 2024-06-18T13:07:02.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:01 smithi167 ceph-mon[37701]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:02.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi167-73976-31"}]': finished 2024-06-18T13:07:02.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:02.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:01 smithi167 ceph-mon[37701]: osdmap e195: 8 total, 8 up, 8 in 2024-06-18T13:07:02.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:07:02.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19"}]: dispatch 2024-06-18T13:07:02.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19"}]: dispatch 2024-06-18T13:07:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:01 smithi167 ceph-mon[29496]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi167-73976-31"}]': finished 2024-06-18T13:07:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:01 smithi167 ceph-mon[29496]: osdmap e195: 8 total, 8 up, 8 in 2024-06-18T13:07:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi167-73976-31"}]: dispatch 2024-06-18T13:07:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19"}]: dispatch 2024-06-18T13:07:02.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19"}]: dispatch 2024-06-18T13:07:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:02 smithi179 ceph-mon[33312]: pgmap v245: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:02 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi167-73976-31"}]': finished 2024-06-18T13:07:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19"}]': finished 2024-06-18T13:07:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:02 smithi179 ceph-mon[33312]: osdmap e196: 8 total, 8 up, 8 in 2024-06-18T13:07:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:07:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3117757456' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi167-73991-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi167-73991-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi167-73976-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:03.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi167-73976-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[37701]: pgmap v245: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi167-73976-31"}]': finished 2024-06-18T13:07:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19"}]': finished 2024-06-18T13:07:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[37701]: osdmap e196: 8 total, 8 up, 8 in 2024-06-18T13:07:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:07:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3117757456' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi167-73991-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi167-73991-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi167-73976-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi167-73976-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:03.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[29496]: pgmap v245: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:03.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:03.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3790924005' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi167-73976-31"}]': finished 2024-06-18T13:07:03.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-19"}]': finished 2024-06-18T13:07:03.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[29496]: osdmap e196: 8 total, 8 up, 8 in 2024-06-18T13:07:03.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:07:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3117757456' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi167-73991-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi167-73991-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi167-73976-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:03.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi167-73976-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:04.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi167-74075-2"}]': finished 2024-06-18T13:07:04.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi167-73991-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:04.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi167-73976-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:04.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:04 smithi179 ceph-mon[33312]: osdmap e197: 8 total, 8 up, 8 in 2024-06-18T13:07:04.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi167-73976-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:04.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:07:04.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi167-73976-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:04.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:04 smithi179 ceph-mon[33312]: pgmap v248: 292 pgs: 292 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:07:04.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi167-74075-2"}]': finished 2024-06-18T13:07:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi167-73991-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi167-73976-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[37701]: osdmap e197: 8 total, 8 up, 8 in 2024-06-18T13:07:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi167-73976-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:07:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi167-73976-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[37701]: pgmap v248: 292 pgs: 292 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:07:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi167-74075-2"}]': finished 2024-06-18T13:07:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi167-73991-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi167-73976-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[29496]: osdmap e197: 8 total, 8 up, 8 in 2024-06-18T13:07:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi167-73976-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi167-74075-2"}]: dispatch 2024-06-18T13:07:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi167-73976-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:04 smithi167 ceph-mon[29496]: pgmap v248: 292 pgs: 292 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:07:05.427 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:07:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:07:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:07:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi167-74075-2"}]': finished 2024-06-18T13:07:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:05 smithi167 ceph-mon[29496]: osdmap e198: 8 total, 8 up, 8 in 2024-06-18T13:07:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:05.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi167-74075-2"}]': finished 2024-06-18T13:07:05.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:05 smithi167 ceph-mon[37701]: osdmap e198: 8 total, 8 up, 8 in 2024-06-18T13:07:05.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:05.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:05.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3253257470' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi167-74075-2"}]': finished 2024-06-18T13:07:05.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:05 smithi179 ceph-mon[33312]: osdmap e198: 8 total, 8 up, 8 in 2024-06-18T13:07:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:06.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi167-73976-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi167-73976-32"}]': finished 2024-06-18T13:07:06.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:06.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[29496]: osdmap e199: 8 total, 8 up, 8 in 2024-06-18T13:07:06.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3113226709' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:06.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1330789031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74075-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:06.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:06.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:06.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:06.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[29496]: pgmap v251: 364 pgs: 104 unknown, 260 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:07:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi167-73976-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi167-73976-32"}]': finished 2024-06-18T13:07:06.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:06.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[37701]: osdmap e199: 8 total, 8 up, 8 in 2024-06-18T13:07:06.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3113226709' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:06.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1330789031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74075-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:06.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:06.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:06.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:06.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:06 smithi167 ceph-mon[37701]: pgmap v251: 364 pgs: 104 unknown, 260 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:07:06.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi167-73976-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi167-73976-32"}]': finished 2024-06-18T13:07:06.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:06 smithi179 ceph-mon[33312]: osdmap e199: 8 total, 8 up, 8 in 2024-06-18T13:07:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3113226709' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1330789031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74075-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:06 smithi179 ceph-mon[33312]: pgmap v251: 364 pgs: 104 unknown, 260 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:07:07.473 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-06-18T13:07:07.473 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 1 0 2024-06-18T13:07:07.473 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 10 0 2024-06-18T13:07:07.473 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 13 0 2024-06-18T13:07:07.473 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 7 0 2024-06-18T13:07:07.473 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 14 0 2024-06-18T13:07:07.473 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 0 0 2024-06-18T13:07:07.473 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 15 0 2024-06-18T13:07:07.473 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 11 0 2024-06-18T13:07:07.474 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 5 0 2024-06-18T13:07:07.474 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 8 0 2024-06-18T13:07:07.474 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 6 0 2024-06-18T13:07:07.474 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 3 0 2024-06-18T13:07:07.474 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 4 0 2024-06-18T13:07:07.474 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 12 0 2024-06-18T13:07:07.474 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 9 0 2024-06-18T13:07:07.474 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 2 0 2024-06-18T13:07:07.474 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-06-18T13:07:07.474 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (44 ms) 2024-06-18T13:07:07.474 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [----------] 3 tests from LibRadosListEC (1225 ms total) 2024-06-18T13:07:07.474 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 2024-06-18T13:07:07.474 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-06-18T13:07:07.475 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-06-18T13:07:07.475 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (2796 ms) 2024-06-18T13:07:07.475 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [----------] 1 test from LibRadosListNP (2796 ms total) 2024-06-18T13:07:07.475 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: 2024-06-18T13:07:07.475 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [----------] Global test environment tear-down 2024-06-18T13:07:07.475 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [==========] 11 tests from 3 test suites ran. (133383 ms total) 2024-06-18T13:07:07.475 INFO:tasks.workunit.client.0.smithi167.stdout: api_list: [ PASSED ] 11 tests. 2024-06-18T13:07:07.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[37701]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:07.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1330789031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74075-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:07.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:07.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:07.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[37701]: osdmap e200: 8 total, 8 up, 8 in 2024-06-18T13:07:07.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-21"}]: dispatch 2024-06-18T13:07:07.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-21"}]: dispatch 2024-06-18T13:07:07.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1330789031' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi167-74075-3","pool2":"test-rados-api-smithi167-74075-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-06-18T13:07:07.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[29496]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:07.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1330789031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74075-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:07.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:07.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:07.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[29496]: osdmap e200: 8 total, 8 up, 8 in 2024-06-18T13:07:07.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-21"}]: dispatch 2024-06-18T13:07:07.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-21"}]: dispatch 2024-06-18T13:07:07.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1330789031' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi167-74075-3","pool2":"test-rados-api-smithi167-74075-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-06-18T13:07:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:07 smithi179 ceph-mon[33312]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1330789031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74075-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:07 smithi179 ceph-mon[33312]: osdmap e200: 8 total, 8 up, 8 in 2024-06-18T13:07:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-21"}]: dispatch 2024-06-18T13:07:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-21"}]: dispatch 2024-06-18T13:07:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1330789031' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi167-74075-3","pool2":"test-rados-api-smithi167-74075-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-06-18T13:07:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-21"}]': finished 2024-06-18T13:07:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1330789031' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi167-74075-3","pool2":"test-rados-api-smithi167-74075-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-06-18T13:07:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: osdmap e201: 8 total, 8 up, 8 in 2024-06-18T13:07:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-21", "mode": "writeback"}]: dispatch 2024-06-18T13:07:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2130496845' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-21", "mode": "writeback"}]: dispatch 2024-06-18T13:07:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: pgmap v254: 356 pgs: 96 unknown, 260 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:08.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:08.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-21", "mode": "writeback"}]': finished 2024-06-18T13:07:08.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]': finished 2024-06-18T13:07:08.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:08.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: osdmap e202: 8 total, 8 up, 8 in 2024-06-18T13:07:08.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:08.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:08.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-21"}]': finished 2024-06-18T13:07:08.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1330789031' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi167-74075-3","pool2":"test-rados-api-smithi167-74075-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-06-18T13:07:08.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: osdmap e201: 8 total, 8 up, 8 in 2024-06-18T13:07:08.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-21", "mode": "writeback"}]: dispatch 2024-06-18T13:07:08.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:08.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2130496845' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:08.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-21", "mode": "writeback"}]: dispatch 2024-06-18T13:07:08.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:08.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:08.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: pgmap v254: 356 pgs: 96 unknown, 260 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:08.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:08.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-21", "mode": "writeback"}]': finished 2024-06-18T13:07:08.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]': finished 2024-06-18T13:07:08.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:08.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: osdmap e202: 8 total, 8 up, 8 in 2024-06-18T13:07:08.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:08.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:08.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:07:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-21"}]': finished 2024-06-18T13:07:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1330789031' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi167-74075-3","pool2":"test-rados-api-smithi167-74075-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-06-18T13:07:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: osdmap e201: 8 total, 8 up, 8 in 2024-06-18T13:07:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-21", "mode": "writeback"}]: dispatch 2024-06-18T13:07:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2130496845' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:08.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-21", "mode": "writeback"}]: dispatch 2024-06-18T13:07:08.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:08.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:08.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: pgmap v254: 356 pgs: 96 unknown, 260 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:08.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:08.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-21", "mode": "writeback"}]': finished 2024-06-18T13:07:08.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi167-73976-32"}]': finished 2024-06-18T13:07:08.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:08.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: osdmap e202: 8 total, 8 up, 8 in 2024-06-18T13:07:08.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1953239348' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:08.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]: dispatch 2024-06-18T13:07:09.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:07:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]': finished 2024-06-18T13:07:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[37701]: osdmap e203: 8 total, 8 up, 8 in 2024-06-18T13:07:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21"}]: dispatch 2024-06-18T13:07:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2881941320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21"}]: dispatch 2024-06-18T13:07:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:07:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]': finished 2024-06-18T13:07:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[29496]: osdmap e203: 8 total, 8 up, 8 in 2024-06-18T13:07:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21"}]: dispatch 2024-06-18T13:07:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2881941320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21"}]: dispatch 2024-06-18T13:07:09.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:09 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:07:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi167-73976-32"}]': finished 2024-06-18T13:07:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:09 smithi179 ceph-mon[33312]: osdmap e203: 8 total, 8 up, 8 in 2024-06-18T13:07:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21"}]: dispatch 2024-06-18T13:07:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2881941320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21"}]: dispatch 2024-06-18T13:07:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi167-73976-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi167-73976-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:10 smithi179 ceph-mon[33312]: pgmap v257: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-18T13:07:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:10 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2881941320' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21"}]': finished 2024-06-18T13:07:10.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi167-73976-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:10.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:10 smithi179 ceph-mon[33312]: osdmap e204: 8 total, 8 up, 8 in 2024-06-18T13:07:10.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi167-73976-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi167-73976-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi167-73976-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi167-73976-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[29496]: pgmap v257: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-18T13:07:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:10.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2881941320' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:10.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21"}]': finished 2024-06-18T13:07:10.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi167-73976-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:10.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[29496]: osdmap e204: 8 total, 8 up, 8 in 2024-06-18T13:07:10.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi167-73976-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi167-73976-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi167-73976-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:10.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi167-73976-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:10.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[37701]: pgmap v257: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-18T13:07:10.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:10.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2881941320' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi167-73991-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:10.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-21"}]': finished 2024-06-18T13:07:10.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi167-73976-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:10.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[37701]: osdmap e204: 8 total, 8 up, 8 in 2024-06-18T13:07:10.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi167-73976-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:10.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi167-73976-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:12.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:12 smithi167 ceph-mon[37701]: osdmap e205: 8 total, 8 up, 8 in 2024-06-18T13:07:12.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:12 smithi167 ceph-mon[37701]: pgmap v260: 324 pgs: 324 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-18T13:07:12.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:12 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:12 smithi167 ceph-mon[29496]: osdmap e205: 8 total, 8 up, 8 in 2024-06-18T13:07:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:12 smithi167 ceph-mon[29496]: pgmap v260: 324 pgs: 324 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-18T13:07:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:12 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:12 smithi179 ceph-mon[33312]: osdmap e205: 8 total, 8 up, 8 in 2024-06-18T13:07:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:12 smithi179 ceph-mon[33312]: pgmap v260: 324 pgs: 324 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-18T13:07:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:12 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:13.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi167-73976-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi167-73976-33"}]': finished 2024-06-18T13:07:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:13 smithi167 ceph-mon[37701]: osdmap e206: 8 total, 8 up, 8 in 2024-06-18T13:07:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi167-73976-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi167-73976-33"}]': finished 2024-06-18T13:07:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:13 smithi167 ceph-mon[29496]: osdmap e206: 8 total, 8 up, 8 in 2024-06-18T13:07:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:13.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi167-73976-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi167-73976-33"}]': finished 2024-06-18T13:07:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:13 smithi179 ceph-mon[33312]: osdmap e206: 8 total, 8 up, 8 in 2024-06-18T13:07:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:14.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:14 smithi179 ceph-mon[33312]: osdmap e207: 8 total, 8 up, 8 in 2024-06-18T13:07:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:14 smithi179 ceph-mon[33312]: pgmap v263: 300 pgs: 2 active, 16 creating+activating, 282 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:07:14.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:14 smithi167 ceph-mon[37701]: osdmap e207: 8 total, 8 up, 8 in 2024-06-18T13:07:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:14 smithi167 ceph-mon[37701]: pgmap v263: 300 pgs: 2 active, 16 creating+activating, 282 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:07:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:14 smithi167 ceph-mon[29496]: osdmap e207: 8 total, 8 up, 8 in 2024-06-18T13:07:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:14 smithi167 ceph-mon[29496]: pgmap v263: 300 pgs: 2 active, 16 creating+activating, 282 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:07:15.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:07:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:07:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:15 smithi179 ceph-mon[33312]: osdmap e208: 8 total, 8 up, 8 in 2024-06-18T13:07:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/421583768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:15.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-mon[37701]: osdmap e208: 8 total, 8 up, 8 in 2024-06-18T13:07:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/421583768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-mon[29496]: osdmap e208: 8 total, 8 up, 8 in 2024-06-18T13:07:15.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/421583768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:15.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:15.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:15.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:15.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:15.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:16.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]': finished 2024-06-18T13:07:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: osdmap e209: 8 total, 8 up, 8 in 2024-06-18T13:07:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-23"}]: dispatch 2024-06-18T13:07:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-23"}]: dispatch 2024-06-18T13:07:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: pgmap v266: 324 pgs: 32 unknown, 1 active, 12 creating+activating, 279 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]': finished 2024-06-18T13:07:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-23"}]': finished 2024-06-18T13:07:16.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: osdmap e210: 8 total, 8 up, 8 in 2024-06-18T13:07:16.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-23", "mode": "writeback"}]: dispatch 2024-06-18T13:07:16.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1599836245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:16.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-23", "mode": "writeback"}]: dispatch 2024-06-18T13:07:16.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]': finished 2024-06-18T13:07:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: osdmap e209: 8 total, 8 up, 8 in 2024-06-18T13:07:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-23"}]: dispatch 2024-06-18T13:07:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:16.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-23"}]: dispatch 2024-06-18T13:07:16.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: pgmap v266: 324 pgs: 32 unknown, 1 active, 12 creating+activating, 279 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:16.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]': finished 2024-06-18T13:07:16.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-23"}]': finished 2024-06-18T13:07:16.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: osdmap e210: 8 total, 8 up, 8 in 2024-06-18T13:07:16.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-23", "mode": "writeback"}]: dispatch 2024-06-18T13:07:16.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1599836245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:16.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-23", "mode": "writeback"}]: dispatch 2024-06-18T13:07:16.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:16.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:16.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi167-73976-33"}]': finished 2024-06-18T13:07:16.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:16.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: osdmap e209: 8 total, 8 up, 8 in 2024-06-18T13:07:16.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1941693164' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:16.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-23"}]: dispatch 2024-06-18T13:07:16.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]: dispatch 2024-06-18T13:07:16.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-23"}]: dispatch 2024-06-18T13:07:16.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: pgmap v266: 324 pgs: 32 unknown, 1 active, 12 creating+activating, 279 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:16.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi167-73976-33"}]': finished 2024-06-18T13:07:16.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-23"}]': finished 2024-06-18T13:07:16.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: osdmap e210: 8 total, 8 up, 8 in 2024-06-18T13:07:16.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-23", "mode": "writeback"}]: dispatch 2024-06-18T13:07:16.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1599836245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:16.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-23", "mode": "writeback"}]: dispatch 2024-06-18T13:07:16.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi167-73976-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:17.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi167-73976-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:17.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:17 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:17.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:17 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:17.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-23", "mode": "writeback"}]': finished 2024-06-18T13:07:17.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:17.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi167-73976-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:17.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:17 smithi179 ceph-mon[33312]: osdmap e211: 8 total, 8 up, 8 in 2024-06-18T13:07:17.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi167-73976-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi167-73976-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi167-73976-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi167-73976-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-23", "mode": "writeback"}]': finished 2024-06-18T13:07:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:17.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi167-73976-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:17.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[29496]: osdmap e211: 8 total, 8 up, 8 in 2024-06-18T13:07:17.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi167-73976-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi167-73976-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi167-73976-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:17.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi167-73976-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:17.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:17.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:17.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-23", "mode": "writeback"}]': finished 2024-06-18T13:07:17.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:17.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi167-73976-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:17.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[37701]: osdmap e211: 8 total, 8 up, 8 in 2024-06-18T13:07:17.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi167-73976-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:17.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi167-73976-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:19.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:18 smithi179 ceph-mon[33312]: pgmap v269: 356 pgs: 64 unknown, 1 active, 12 creating+activating, 279 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:19.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:07:18 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:07:19.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:18 smithi167 ceph-mon[29496]: pgmap v269: 356 pgs: 64 unknown, 1 active, 12 creating+activating, 279 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:19.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:18 smithi167 ceph-mon[37701]: pgmap v269: 356 pgs: 64 unknown, 1 active, 12 creating+activating, 279 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:20.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:19 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:19 smithi179 ceph-mon[33312]: osdmap e212: 8 total, 8 up, 8 in 2024-06-18T13:07:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2202177972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi167-73976-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi167-73976-34"}]': finished 2024-06-18T13:07:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2202177972' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:19 smithi179 ceph-mon[33312]: osdmap e213: 8 total, 8 up, 8 in 2024-06-18T13:07:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23"}]: dispatch 2024-06-18T13:07:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23"}]: dispatch 2024-06-18T13:07:20.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:20.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[37701]: osdmap e212: 8 total, 8 up, 8 in 2024-06-18T13:07:20.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2202177972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:20.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:20.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:20.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi167-73976-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi167-73976-34"}]': finished 2024-06-18T13:07:20.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2202177972' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:20.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:20.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[37701]: osdmap e213: 8 total, 8 up, 8 in 2024-06-18T13:07:20.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23"}]: dispatch 2024-06-18T13:07:20.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23"}]: dispatch 2024-06-18T13:07:20.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:20.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[29496]: osdmap e212: 8 total, 8 up, 8 in 2024-06-18T13:07:20.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2202177972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:20.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:20.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:20.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi167-73976-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi167-73976-34"}]': finished 2024-06-18T13:07:20.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2202177972' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:20.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:20.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[29496]: osdmap e213: 8 total, 8 up, 8 in 2024-06-18T13:07:20.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23"}]: dispatch 2024-06-18T13:07:20.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23"}]: dispatch 2024-06-18T13:07:21.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:20 smithi179 ceph-mon[33312]: pgmap v272: 396 pgs: 5 creating+peering, 1 active+clean+snaptrim, 35 unknown, 355 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-06-18T13:07:21.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:20 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23"}]': finished 2024-06-18T13:07:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:20 smithi179 ceph-mon[33312]: osdmap e214: 8 total, 8 up, 8 in 2024-06-18T13:07:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2666931117' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:21.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:20 smithi167 ceph-mon[37701]: pgmap v272: 396 pgs: 5 creating+peering, 1 active+clean+snaptrim, 35 unknown, 355 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-06-18T13:07:21.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:20 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:21.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23"}]': finished 2024-06-18T13:07:21.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:20 smithi167 ceph-mon[37701]: osdmap e214: 8 total, 8 up, 8 in 2024-06-18T13:07:21.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2666931117' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:21.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:20 smithi167 ceph-mon[29496]: pgmap v272: 396 pgs: 5 creating+peering, 1 active+clean+snaptrim, 35 unknown, 355 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-06-18T13:07:21.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:20 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:21.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-23"}]': finished 2024-06-18T13:07:21.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:20 smithi167 ceph-mon[29496]: osdmap e214: 8 total, 8 up, 8 in 2024-06-18T13:07:21.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2666931117' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:22.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2666931117' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:22 smithi179 ceph-mon[33312]: osdmap e215: 8 total, 8 up, 8 in 2024-06-18T13:07:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:22 smithi179 ceph-mon[33312]: pgmap v275: 388 pgs: 5 creating+peering, 1 active+clean+snaptrim, 59 unknown, 323 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-06-18T13:07:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:22 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2666931117' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:22 smithi167 ceph-mon[37701]: osdmap e215: 8 total, 8 up, 8 in 2024-06-18T13:07:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:22 smithi167 ceph-mon[37701]: pgmap v275: 388 pgs: 5 creating+peering, 1 active+clean+snaptrim, 59 unknown, 323 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-06-18T13:07:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:22 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2666931117' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:22 smithi167 ceph-mon[29496]: osdmap e215: 8 total, 8 up, 8 in 2024-06-18T13:07:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:22 smithi167 ceph-mon[29496]: pgmap v275: 388 pgs: 5 creating+peering, 1 active+clean+snaptrim, 59 unknown, 323 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-06-18T13:07:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:22 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:23.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]': finished 2024-06-18T13:07:23.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:23 smithi179 ceph-mon[33312]: osdmap e216: 8 total, 8 up, 8 in 2024-06-18T13:07:23.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:23.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:23.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2804520267' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:23.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:23.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:23.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]': finished 2024-06-18T13:07:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[37701]: osdmap e216: 8 total, 8 up, 8 in 2024-06-18T13:07:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:23.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:23.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2804520267' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:23.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:23.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:23.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:23.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi167-73976-34"}]': finished 2024-06-18T13:07:23.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[29496]: osdmap e216: 8 total, 8 up, 8 in 2024-06-18T13:07:23.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4036634122' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:23.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:23.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2804520267' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:23.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]: dispatch 2024-06-18T13:07:23.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:23.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]': finished 2024-06-18T13:07:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: osdmap e217: 8 total, 8 up, 8 in 2024-06-18T13:07:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi167-73976-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi167-73976-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: pgmap v278: 452 pgs: 19 creating+activating, 40 creating+peering, 393 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:24.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:07:24.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:07:24.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi167-73976-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:24.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: osdmap e218: 8 total, 8 up, 8 in 2024-06-18T13:07:24.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi167-73976-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi167-73976-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:24.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:24.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]': finished 2024-06-18T13:07:24.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:24.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:24.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: osdmap e217: 8 total, 8 up, 8 in 2024-06-18T13:07:24.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi167-73976-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:24.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi167-73976-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:24.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: pgmap v278: 452 pgs: 19 creating+activating, 40 creating+peering, 393 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:24.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:07:24.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:07:24.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi167-73976-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:24.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: osdmap e218: 8 total, 8 up, 8 in 2024-06-18T13:07:24.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi167-73976-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi167-73976-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:24.981 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:24.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi167-73976-34"}]': finished 2024-06-18T13:07:24.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:24.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi167-73991-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:24.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: osdmap e217: 8 total, 8 up, 8 in 2024-06-18T13:07:24.982 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.982 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.982 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.982 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.983 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi167-73976-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:24.983 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi167-73976-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:24.983 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: pgmap v278: 452 pgs: 19 creating+activating, 40 creating+peering, 393 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:24.983 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:07:24.983 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:07:24.983 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi167-73976-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:24.983 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: osdmap e218: 8 total, 8 up, 8 in 2024-06-18T13:07:24.983 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi167-73976-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.984 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi167-73976-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:24.984 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:24.984 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:25.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:07:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:07:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:07:26.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-25"}]: dispatch 2024-06-18T13:07:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:26 smithi179 ceph-mon[33312]: osdmap e219: 8 total, 8 up, 8 in 2024-06-18T13:07:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-25"}]: dispatch 2024-06-18T13:07:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:26 smithi179 ceph-mon[33312]: pgmap v281: 388 pgs: 8 creating+activating, 21 creating+peering, 359 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:07:26.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-25"}]: dispatch 2024-06-18T13:07:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:26 smithi167 ceph-mon[37701]: osdmap e219: 8 total, 8 up, 8 in 2024-06-18T13:07:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-25"}]: dispatch 2024-06-18T13:07:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:26 smithi167 ceph-mon[37701]: pgmap v281: 388 pgs: 8 creating+activating, 21 creating+peering, 359 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:07:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-25"}]: dispatch 2024-06-18T13:07:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:26 smithi167 ceph-mon[29496]: osdmap e219: 8 total, 8 up, 8 in 2024-06-18T13:07:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-25"}]: dispatch 2024-06-18T13:07:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:26 smithi167 ceph-mon[29496]: pgmap v281: 388 pgs: 8 creating+activating, 21 creating+peering, 359 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:07:27.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi167-73976-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi167-73976-35"}]': finished 2024-06-18T13:07:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-25"}]': finished 2024-06-18T13:07:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:27 smithi179 ceph-mon[33312]: osdmap e220: 8 total, 8 up, 8 in 2024-06-18T13:07:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-25", "mode": "writeback"}]: dispatch 2024-06-18T13:07:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-25", "mode": "writeback"}]: dispatch 2024-06-18T13:07:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:27 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:27 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-25", "mode": "writeback"}]': finished 2024-06-18T13:07:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:27 smithi179 ceph-mon[33312]: osdmap e221: 8 total, 8 up, 8 in 2024-06-18T13:07:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:27.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi167-73976-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi167-73976-35"}]': finished 2024-06-18T13:07:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-25"}]': finished 2024-06-18T13:07:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[37701]: osdmap e220: 8 total, 8 up, 8 in 2024-06-18T13:07:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-25", "mode": "writeback"}]: dispatch 2024-06-18T13:07:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-25", "mode": "writeback"}]: dispatch 2024-06-18T13:07:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-25", "mode": "writeback"}]': finished 2024-06-18T13:07:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[37701]: osdmap e221: 8 total, 8 up, 8 in 2024-06-18T13:07:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi167-73976-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi167-73976-35"}]': finished 2024-06-18T13:07:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-25"}]': finished 2024-06-18T13:07:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[29496]: osdmap e220: 8 total, 8 up, 8 in 2024-06-18T13:07:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-25", "mode": "writeback"}]: dispatch 2024-06-18T13:07:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-25", "mode": "writeback"}]: dispatch 2024-06-18T13:07:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-25", "mode": "writeback"}]': finished 2024-06-18T13:07:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[29496]: osdmap e221: 8 total, 8 up, 8 in 2024-06-18T13:07:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:29.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:07:28 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:07:29.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:29 smithi167 ceph-mon[37701]: pgmap v284: 332 pgs: 8 unknown, 8 creating+activating, 21 creating+peering, 295 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:29 smithi167 ceph-mon[37701]: osdmap e222: 8 total, 8 up, 8 in 2024-06-18T13:07:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25"}]: dispatch 2024-06-18T13:07:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25"}]: dispatch 2024-06-18T13:07:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:29 smithi167 ceph-mon[29496]: pgmap v284: 332 pgs: 8 unknown, 8 creating+activating, 21 creating+peering, 295 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:29 smithi167 ceph-mon[29496]: osdmap e222: 8 total, 8 up, 8 in 2024-06-18T13:07:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25"}]: dispatch 2024-06-18T13:07:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25"}]: dispatch 2024-06-18T13:07:29.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:29 smithi179 ceph-mon[33312]: pgmap v284: 332 pgs: 8 unknown, 8 creating+activating, 21 creating+peering, 295 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:29 smithi179 ceph-mon[33312]: osdmap e222: 8 total, 8 up, 8 in 2024-06-18T13:07:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25"}]: dispatch 2024-06-18T13:07:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25"}]: dispatch 2024-06-18T13:07:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25"}]': finished 2024-06-18T13:07:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]': finished 2024-06-18T13:07:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[29496]: osdmap e223: 8 total, 8 up, 8 in 2024-06-18T13:07:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2309507987' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi167-73991-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi167-73991-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[29496]: pgmap v287: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:07:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]': finished 2024-06-18T13:07:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi167-73991-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[29496]: osdmap e224: 8 total, 8 up, 8 in 2024-06-18T13:07:30.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:30.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:30.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25"}]': finished 2024-06-18T13:07:30.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]': finished 2024-06-18T13:07:30.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[37701]: osdmap e223: 8 total, 8 up, 8 in 2024-06-18T13:07:30.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:30.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:30.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2309507987' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi167-73991-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:30.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi167-73991-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:30.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[37701]: pgmap v287: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:07:30.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]': finished 2024-06-18T13:07:30.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi167-73991-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:30.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:30 smithi167 ceph-mon[37701]: osdmap e224: 8 total, 8 up, 8 in 2024-06-18T13:07:31.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:30 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-25"}]': finished 2024-06-18T13:07:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi167-73976-35"}]': finished 2024-06-18T13:07:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:30 smithi179 ceph-mon[33312]: osdmap e223: 8 total, 8 up, 8 in 2024-06-18T13:07:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/430458440' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]: dispatch 2024-06-18T13:07:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2309507987' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi167-73991-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi167-73991-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:30 smithi179 ceph-mon[33312]: pgmap v287: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:07:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi167-73976-35"}]': finished 2024-06-18T13:07:31.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi167-73991-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:31.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:30 smithi179 ceph-mon[33312]: osdmap e224: 8 total, 8 up, 8 in 2024-06-18T13:07:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi167-73976-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi167-73976-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi167-73976-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi167-73976-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:31.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[37701]: osdmap e225: 8 total, 8 up, 8 in 2024-06-18T13:07:31.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:31.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi167-73976-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:31.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:31.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:31.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:31.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:31.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:31.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi167-73976-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:31.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi167-73976-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:31.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi167-73976-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:31.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi167-73976-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:31.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[29496]: osdmap e225: 8 total, 8 up, 8 in 2024-06-18T13:07:31.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:31.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi167-73976-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:31.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:32.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi167-73976-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi167-73976-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi167-73976-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi167-73976-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:31 smithi179 ceph-mon[33312]: osdmap e225: 8 total, 8 up, 8 in 2024-06-18T13:07:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi167-73976-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:32 smithi167 ceph-mon[37701]: pgmap v290: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:32 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:32 smithi167 ceph-mon[37701]: osdmap e226: 8 total, 8 up, 8 in 2024-06-18T13:07:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2669740545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi167-73991-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi167-73991-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:32 smithi167 ceph-mon[29496]: pgmap v290: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:32 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:32.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:32 smithi167 ceph-mon[29496]: osdmap e226: 8 total, 8 up, 8 in 2024-06-18T13:07:32.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2669740545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi167-73991-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:32.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi167-73991-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:33.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:32 smithi179 ceph-mon[33312]: pgmap v290: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:32 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:32 smithi179 ceph-mon[33312]: osdmap e226: 8 total, 8 up, 8 in 2024-06-18T13:07:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2669740545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi167-73991-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi167-73991-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:34.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi167-73976-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi167-73976-36"}]': finished 2024-06-18T13:07:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi167-73991-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:34 smithi167 ceph-mon[37701]: osdmap e227: 8 total, 8 up, 8 in 2024-06-18T13:07:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:34 smithi167 ceph-mon[37701]: pgmap v293: 332 pgs: 8 creating+peering, 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-06-18T13:07:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi167-73976-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi167-73976-36"}]': finished 2024-06-18T13:07:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi167-73991-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:34 smithi167 ceph-mon[29496]: osdmap e227: 8 total, 8 up, 8 in 2024-06-18T13:07:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:34.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:34 smithi167 ceph-mon[29496]: pgmap v293: 332 pgs: 8 creating+peering, 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-06-18T13:07:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi167-73976-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi167-73976-36"}]': finished 2024-06-18T13:07:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi167-73991-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:34 smithi179 ceph-mon[33312]: osdmap e227: 8 total, 8 up, 8 in 2024-06-18T13:07:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:34 smithi179 ceph-mon[33312]: pgmap v293: 332 pgs: 8 creating+peering, 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-06-18T13:07:35.432 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:07:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:07:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[29496]: osdmap e228: 8 total, 8 up, 8 in 2024-06-18T13:07:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-27"}]: dispatch 2024-06-18T13:07:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-27"}]: dispatch 2024-06-18T13:07:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:35.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:35.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:35.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:35.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-27"}]': finished 2024-06-18T13:07:35.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:35.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[37701]: osdmap e228: 8 total, 8 up, 8 in 2024-06-18T13:07:35.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-27"}]: dispatch 2024-06-18T13:07:35.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-27"}]: dispatch 2024-06-18T13:07:35.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:35.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:35.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:35.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:35.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:35.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:35.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-27"}]': finished 2024-06-18T13:07:35.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:35 smithi179 ceph-mon[33312]: osdmap e228: 8 total, 8 up, 8 in 2024-06-18T13:07:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-27"}]: dispatch 2024-06-18T13:07:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-27"}]: dispatch 2024-06-18T13:07:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-27"}]': finished 2024-06-18T13:07:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[37701]: osdmap e229: 8 total, 8 up, 8 in 2024-06-18T13:07:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-27", "mode": "writeback"}]: dispatch 2024-06-18T13:07:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-27", "mode": "writeback"}]: dispatch 2024-06-18T13:07:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:36.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[37701]: pgmap v296: 292 pgs: 292 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-18T13:07:36.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:36.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]': finished 2024-06-18T13:07:36.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:36.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[29496]: osdmap e229: 8 total, 8 up, 8 in 2024-06-18T13:07:36.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:36.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:36.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-27", "mode": "writeback"}]: dispatch 2024-06-18T13:07:36.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:36.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-27", "mode": "writeback"}]: dispatch 2024-06-18T13:07:36.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:36.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[29496]: pgmap v296: 292 pgs: 292 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-18T13:07:36.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:36.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]': finished 2024-06-18T13:07:37.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:37.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:36 smithi179 ceph-mon[33312]: osdmap e229: 8 total, 8 up, 8 in 2024-06-18T13:07:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-27", "mode": "writeback"}]: dispatch 2024-06-18T13:07:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-27", "mode": "writeback"}]: dispatch 2024-06-18T13:07:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:36 smithi179 ceph-mon[33312]: pgmap v296: 292 pgs: 292 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-18T13:07:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:36 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi167-73976-36"}]': finished 2024-06-18T13:07:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-27", "mode": "writeback"}]': finished 2024-06-18T13:07:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: osdmap e230: 8 total, 8 up, 8 in 2024-06-18T13:07:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-46"}]': finished 2024-06-18T13:07:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]': finished 2024-06-18T13:07:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: osdmap e231: 8 total, 8 up, 8 in 2024-06-18T13:07:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:37.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:37.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-27", "mode": "writeback"}]': finished 2024-06-18T13:07:37.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:37.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: osdmap e230: 8 total, 8 up, 8 in 2024-06-18T13:07:37.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:37.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:37.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-46"}]': finished 2024-06-18T13:07:37.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]': finished 2024-06-18T13:07:37.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: osdmap e231: 8 total, 8 up, 8 in 2024-06-18T13:07:37.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:37.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:37.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:37.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:37.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:37.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:37.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:37.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-27", "mode": "writeback"}]': finished 2024-06-18T13:07:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3128247990' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: osdmap e230: 8 total, 8 up, 8 in 2024-06-18T13:07:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]: dispatch 2024-06-18T13:07:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-46"}]': finished 2024-06-18T13:07:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi167-73976-36"}]': finished 2024-06-18T13:07:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: osdmap e231: 8 total, 8 up, 8 in 2024-06-18T13:07:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:38.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:38.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:38.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:38.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:38.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[29496]: pgmap v299: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:07:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[29496]: osdmap e232: 8 total, 8 up, 8 in 2024-06-18T13:07:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27"}]: dispatch 2024-06-18T13:07:38.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27"}]: dispatch 2024-06-18T13:07:38.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi167-73976-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:38.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi167-73976-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:38.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:07:38.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:07:38.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:07:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[37701]: pgmap v299: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:07:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:38.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:38.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[37701]: osdmap e232: 8 total, 8 up, 8 in 2024-06-18T13:07:38.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27"}]: dispatch 2024-06-18T13:07:38.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27"}]: dispatch 2024-06-18T13:07:38.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi167-73976-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:38.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi167-73976-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:38.981 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:07:38.982 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:07:38.982 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:38 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:07:39.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:07:38 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:07:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:38 smithi179 ceph-mon[33312]: pgmap v299: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:38 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:07:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:38 smithi179 ceph-mon[33312]: osdmap e232: 8 total, 8 up, 8 in 2024-06-18T13:07:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:38 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27"}]: dispatch 2024-06-18T13:07:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27"}]: dispatch 2024-06-18T13:07:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:38 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi167-73976-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi167-73976-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:39.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:38 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:07:39.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:38 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:07:39.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:38 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:07:40.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:07:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:07:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:40 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27"}]': finished 2024-06-18T13:07:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:40 smithi179 ceph-mon[33312]: osdmap e233: 8 total, 8 up, 8 in 2024-06-18T13:07:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:40.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:07:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:07:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27"}]': finished 2024-06-18T13:07:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[37701]: osdmap e233: 8 total, 8 up, 8 in 2024-06-18T13:07:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:40.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:07:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:07:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-27"}]': finished 2024-06-18T13:07:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[29496]: osdmap e233: 8 total, 8 up, 8 in 2024-06-18T13:07:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:41.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:41 smithi179 ceph-mon[33312]: pgmap v302: 292 pgs: 292 active+clean; 584 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:07:41.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi167-73976-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi167-73976-37"}]': finished 2024-06-18T13:07:41.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]': finished 2024-06-18T13:07:41.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:41 smithi179 ceph-mon[33312]: osdmap e234: 8 total, 8 up, 8 in 2024-06-18T13:07:41.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:41.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:41 smithi167 ceph-mon[37701]: pgmap v302: 292 pgs: 292 active+clean; 584 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:07:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi167-73976-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi167-73976-37"}]': finished 2024-06-18T13:07:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]': finished 2024-06-18T13:07:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:41 smithi167 ceph-mon[37701]: osdmap e234: 8 total, 8 up, 8 in 2024-06-18T13:07:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:41.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:41 smithi167 ceph-mon[29496]: pgmap v302: 292 pgs: 292 active+clean; 584 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:07:41.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi167-73976-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi167-73976-37"}]': finished 2024-06-18T13:07:41.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-46"}]': finished 2024-06-18T13:07:41.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:41 smithi167 ceph-mon[29496]: osdmap e234: 8 total, 8 up, 8 in 2024-06-18T13:07:41.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/22174439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:41.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]: dispatch 2024-06-18T13:07:42.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]': finished 2024-06-18T13:07:42.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[29496]: osdmap e235: 8 total, 8 up, 8 in 2024-06-18T13:07:42.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:42.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:42.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:42.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:42.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:42.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[29496]: pgmap v305: 300 pgs: 40 unknown, 260 active+clean; 584 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:42.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:42.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:42.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[29496]: osdmap e236: 8 total, 8 up, 8 in 2024-06-18T13:07:42.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:42.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:42.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:42.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]': finished 2024-06-18T13:07:42.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[37701]: osdmap e235: 8 total, 8 up, 8 in 2024-06-18T13:07:42.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:42.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:42.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:42.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:42.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:42.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[37701]: pgmap v305: 300 pgs: 40 unknown, 260 active+clean; 584 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:42.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:42.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:42.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[37701]: osdmap e236: 8 total, 8 up, 8 in 2024-06-18T13:07:42.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:42.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:42.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:42.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-46"}]': finished 2024-06-18T13:07:42.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:42 smithi179 ceph-mon[33312]: osdmap e235: 8 total, 8 up, 8 in 2024-06-18T13:07:42.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:42.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:42.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:42.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:42.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:42.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:42 smithi179 ceph-mon[33312]: pgmap v305: 300 pgs: 40 unknown, 260 active+clean; 584 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:07:42.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:42.890 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi167-73991-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:42.890 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:42 smithi179 ceph-mon[33312]: osdmap e236: 8 total, 8 up, 8 in 2024-06-18T13:07:42.890 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:42.890 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:42.890 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:43.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:43 smithi167 ceph-mon[29496]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]': finished 2024-06-18T13:07:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:43 smithi167 ceph-mon[37701]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]': finished 2024-06-18T13:07:44.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:43 smithi179 ceph-mon[33312]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi167-73976-37"}]': finished 2024-06-18T13:07:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[37701]: osdmap e237: 8 total, 8 up, 8 in 2024-06-18T13:07:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[37701]: pgmap v308: 292 pgs: 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:07:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-47"}]': finished 2024-06-18T13:07:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]': finished 2024-06-18T13:07:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[37701]: osdmap e238: 8 total, 8 up, 8 in 2024-06-18T13:07:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:44.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi167-73976-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:44.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:44.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[29496]: osdmap e237: 8 total, 8 up, 8 in 2024-06-18T13:07:44.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:44.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[29496]: pgmap v308: 292 pgs: 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:07:44.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-47"}]': finished 2024-06-18T13:07:44.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]': finished 2024-06-18T13:07:44.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[29496]: osdmap e238: 8 total, 8 up, 8 in 2024-06-18T13:07:44.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:44.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:44.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:44.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:44.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi167-73976-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2393012033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:44 smithi179 ceph-mon[33312]: osdmap e237: 8 total, 8 up, 8 in 2024-06-18T13:07:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:44 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]: dispatch 2024-06-18T13:07:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:44 smithi179 ceph-mon[33312]: pgmap v308: 292 pgs: 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:07:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi167-73991-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi167-73991-47"}]': finished 2024-06-18T13:07:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:44 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi167-73976-37"}]': finished 2024-06-18T13:07:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:44 smithi179 ceph-mon[33312]: osdmap e238: 8 total, 8 up, 8 in 2024-06-18T13:07:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:44 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi167-73976-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:45.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:07:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:07:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:07:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi167-73976-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:46 smithi167 ceph-mon[37701]: osdmap e239: 8 total, 8 up, 8 in 2024-06-18T13:07:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi167-73976-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-29"}]: dispatch 2024-06-18T13:07:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-29"}]: dispatch 2024-06-18T13:07:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:46 smithi167 ceph-mon[37701]: pgmap v311: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-06-18T13:07:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:46.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi167-73976-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:46.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:46 smithi167 ceph-mon[29496]: osdmap e239: 8 total, 8 up, 8 in 2024-06-18T13:07:46.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi167-73976-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:46.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-29"}]: dispatch 2024-06-18T13:07:46.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-29"}]: dispatch 2024-06-18T13:07:46.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:46 smithi167 ceph-mon[29496]: pgmap v311: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-06-18T13:07:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi167-73976-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:46 smithi179 ceph-mon[33312]: osdmap e239: 8 total, 8 up, 8 in 2024-06-18T13:07:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi167-73976-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-29"}]: dispatch 2024-06-18T13:07:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-29"}]: dispatch 2024-06-18T13:07:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:46 smithi179 ceph-mon[33312]: pgmap v311: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-06-18T13:07:47.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-29"}]': finished 2024-06-18T13:07:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-29", "mode": "writeback"}]: dispatch 2024-06-18T13:07:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:47 smithi167 ceph-mon[37701]: osdmap e240: 8 total, 8 up, 8 in 2024-06-18T13:07:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-29", "mode": "writeback"}]: dispatch 2024-06-18T13:07:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-29"}]': finished 2024-06-18T13:07:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-29", "mode": "writeback"}]: dispatch 2024-06-18T13:07:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:47 smithi167 ceph-mon[29496]: osdmap e240: 8 total, 8 up, 8 in 2024-06-18T13:07:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-29", "mode": "writeback"}]: dispatch 2024-06-18T13:07:47.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:47.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-29"}]': finished 2024-06-18T13:07:47.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-29", "mode": "writeback"}]: dispatch 2024-06-18T13:07:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:47 smithi179 ceph-mon[33312]: osdmap e240: 8 total, 8 up, 8 in 2024-06-18T13:07:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-29", "mode": "writeback"}]: dispatch 2024-06-18T13:07:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:48.690 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:48 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:48.691 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi167-73976-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi167-73976-38"}]': finished 2024-06-18T13:07:48.691 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-29", "mode": "writeback"}]': finished 2024-06-18T13:07:48.691 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-47"}]': finished 2024-06-18T13:07:48.691 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:48 smithi179 ceph-mon[33312]: osdmap e241: 8 total, 8 up, 8 in 2024-06-18T13:07:48.691 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:48.691 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:48 smithi179 ceph-mon[33312]: pgmap v314: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:48.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:48 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi167-73976-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi167-73976-38"}]': finished 2024-06-18T13:07:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-29", "mode": "writeback"}]': finished 2024-06-18T13:07:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-47"}]': finished 2024-06-18T13:07:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:48 smithi167 ceph-mon[37701]: osdmap e241: 8 total, 8 up, 8 in 2024-06-18T13:07:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:48 smithi167 ceph-mon[37701]: pgmap v314: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:48 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi167-73976-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi167-73976-38"}]': finished 2024-06-18T13:07:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-29", "mode": "writeback"}]': finished 2024-06-18T13:07:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi167-73991-47"}]': finished 2024-06-18T13:07:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:48 smithi167 ceph-mon[29496]: osdmap e241: 8 total, 8 up, 8 in 2024-06-18T13:07:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-47"}]: dispatch 2024-06-18T13:07:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:48 smithi167 ceph-mon[29496]: pgmap v314: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:49.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:07:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:07:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-47"}]': finished 2024-06-18T13:07:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[37701]: osdmap e242: 8 total, 8 up, 8 in 2024-06-18T13:07:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi167-73991-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi167-73991-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-47"}]': finished 2024-06-18T13:07:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[29496]: osdmap e242: 8 total, 8 up, 8 in 2024-06-18T13:07:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi167-73991-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi167-73991-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:49 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:49.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1579232499' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi167-73991-47"}]': finished 2024-06-18T13:07:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:49 smithi179 ceph-mon[33312]: osdmap e242: 8 total, 8 up, 8 in 2024-06-18T13:07:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi167-73991-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi167-73991-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:49 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:50.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi167-73991-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:50.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi167-73991-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:50 smithi167 ceph-mon[37701]: osdmap e243: 8 total, 8 up, 8 in 2024-06-18T13:07:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi167-73991-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:50 smithi167 ceph-mon[37701]: pgmap v317: 292 pgs: 292 active+clean; 8.6 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 255 B/s wr, 21 op/s 2024-06-18T13:07:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi167-73991-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi167-73991-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:50 smithi167 ceph-mon[29496]: osdmap e243: 8 total, 8 up, 8 in 2024-06-18T13:07:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi167-73991-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:50 smithi167 ceph-mon[29496]: pgmap v317: 292 pgs: 292 active+clean; 8.6 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 255 B/s wr, 21 op/s 2024-06-18T13:07:50.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi167-73991-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi167-73991-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:50 smithi179 ceph-mon[33312]: osdmap e243: 8 total, 8 up, 8 in 2024-06-18T13:07:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi167-73991-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:50 smithi179 ceph-mon[33312]: pgmap v317: 292 pgs: 292 active+clean; 8.6 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 255 B/s wr, 21 op/s 2024-06-18T13:07:51.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:51 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:51.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:51 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi167-73976-38"}]': finished 2024-06-18T13:07:51.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:51 smithi167 ceph-mon[37701]: osdmap e244: 8 total, 8 up, 8 in 2024-06-18T13:07:51.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:51 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:51.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:51 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:51.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:51 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:51.743 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:51 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:51.743 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:51 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi167-73976-38"}]': finished 2024-06-18T13:07:51.743 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:51 smithi167 ceph-mon[29496]: osdmap e244: 8 total, 8 up, 8 in 2024-06-18T13:07:51.744 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:51 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:51.744 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:51 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:51.744 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:51 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:51.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:51 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:51 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi167-73976-38"}]': finished 2024-06-18T13:07:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:51 smithi179 ceph-mon[33312]: osdmap e244: 8 total, 8 up, 8 in 2024-06-18T13:07:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:51 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi167-73976-38"}]: dispatch 2024-06-18T13:07:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:51 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:51 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:52.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi167-73991-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi167-73991-48"}]': finished 2024-06-18T13:07:52.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi167-73976-38"}]': finished 2024-06-18T13:07:52.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:52.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[37701]: osdmap e245: 8 total, 8 up, 8 in 2024-06-18T13:07:52.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29"}]: dispatch 2024-06-18T13:07:52.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29"}]: dispatch 2024-06-18T13:07:52.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:52.733 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:52.733 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi167-73976-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:52.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[37701]: pgmap v320: 300 pgs: 8 unknown, 292 active+clean; 8.6 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 255 B/s wr, 21 op/s 2024-06-18T13:07:52.734 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi167-73991-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi167-73991-48"}]': finished 2024-06-18T13:07:52.734 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi167-73976-38"}]': finished 2024-06-18T13:07:52.734 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:52.734 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[29496]: osdmap e245: 8 total, 8 up, 8 in 2024-06-18T13:07:52.734 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29"}]: dispatch 2024-06-18T13:07:52.734 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29"}]: dispatch 2024-06-18T13:07:52.734 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:52.735 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:52.735 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi167-73976-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:52.735 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:52 smithi167 ceph-mon[29496]: pgmap v320: 300 pgs: 8 unknown, 292 active+clean; 8.6 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 255 B/s wr, 21 op/s 2024-06-18T13:07:52.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:52 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi167-73991-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi167-73991-48"}]': finished 2024-06-18T13:07:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:52 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2536373441' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi167-73976-38"}]': finished 2024-06-18T13:07:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:52 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:52 smithi179 ceph-mon[33312]: osdmap e245: 8 total, 8 up, 8 in 2024-06-18T13:07:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:52 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29"}]: dispatch 2024-06-18T13:07:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:52 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29"}]: dispatch 2024-06-18T13:07:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:52 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:52 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:52.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:52 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi167-73976-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:52.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:52 smithi179 ceph-mon[33312]: pgmap v320: 300 pgs: 8 unknown, 292 active+clean; 8.6 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 255 B/s wr, 21 op/s 2024-06-18T13:07:53.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:53 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29"}]': finished 2024-06-18T13:07:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi167-73976-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:53 smithi167 ceph-mon[29496]: osdmap e246: 8 total, 8 up, 8 in 2024-06-18T13:07:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi167-73976-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:53.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:53 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:53.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29"}]': finished 2024-06-18T13:07:53.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi167-73976-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:53.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:53 smithi167 ceph-mon[37701]: osdmap e246: 8 total, 8 up, 8 in 2024-06-18T13:07:53.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi167-73976-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:53 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:07:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-29"}]': finished 2024-06-18T13:07:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi167-73976-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:53 smithi179 ceph-mon[33312]: osdmap e246: 8 total, 8 up, 8 in 2024-06-18T13:07:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi167-73976-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:54.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:54 smithi179 ceph-mon[33312]: osdmap e247: 8 total, 8 up, 8 in 2024-06-18T13:07:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:54 smithi179 ceph-mon[33312]: pgmap v323: 260 pgs: 260 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:54 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:07:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:54 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:07:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:54 smithi167 ceph-mon[37701]: osdmap e247: 8 total, 8 up, 8 in 2024-06-18T13:07:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:54 smithi167 ceph-mon[37701]: pgmap v323: 260 pgs: 260 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:54 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:07:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:54 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:07:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:54 smithi167 ceph-mon[29496]: osdmap e247: 8 total, 8 up, 8 in 2024-06-18T13:07:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:54 smithi167 ceph-mon[29496]: pgmap v323: 260 pgs: 260 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:54 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:07:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:54 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:07:55.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:07:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:07:55.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi167-73976-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi167-73976-39"}]': finished 2024-06-18T13:07:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]': finished 2024-06-18T13:07:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:55 smithi179 ceph-mon[33312]: osdmap e248: 8 total, 8 up, 8 in 2024-06-18T13:07:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi167-73976-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi167-73976-39"}]': finished 2024-06-18T13:07:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]': finished 2024-06-18T13:07:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-mon[37701]: osdmap e248: 8 total, 8 up, 8 in 2024-06-18T13:07:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:55.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi167-73976-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi167-73976-39"}]': finished 2024-06-18T13:07:55.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi167-73991-48"}]': finished 2024-06-18T13:07:55.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-mon[29496]: osdmap e248: 8 total, 8 up, 8 in 2024-06-18T13:07:55.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1215472271' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:55.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]: dispatch 2024-06-18T13:07:55.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:55.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:07:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]': finished 2024-06-18T13:07:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:56 smithi179 ceph-mon[33312]: osdmap e249: 8 total, 8 up, 8 in 2024-06-18T13:07:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:56.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:56.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:56.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:56.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:56.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi167-73991-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:56.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi167-73991-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:56.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:56 smithi179 ceph-mon[33312]: pgmap v326: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]': finished 2024-06-18T13:07:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[29496]: osdmap e249: 8 total, 8 up, 8 in 2024-06-18T13:07:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi167-73991-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi167-73991-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[29496]: pgmap v326: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:56.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi167-73991-48"}]': finished 2024-06-18T13:07:56.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:07:56.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[37701]: osdmap e249: 8 total, 8 up, 8 in 2024-06-18T13:07:56.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:56.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:07:56.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:56.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:56.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:56.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:56.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi167-73991-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:56.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi167-73991-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:56.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:56 smithi167 ceph-mon[37701]: pgmap v326: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:57.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi167-73991-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi167-73991-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[29496]: osdmap e250: 8 total, 8 up, 8 in 2024-06-18T13:07:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-31"}]: dispatch 2024-06-18T13:07:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi167-73991-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-31"}]: dispatch 2024-06-18T13:07:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-31"}]': finished 2024-06-18T13:07:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi167-73991-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi167-73991-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[37701]: osdmap e250: 8 total, 8 up, 8 in 2024-06-18T13:07:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-31"}]: dispatch 2024-06-18T13:07:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi167-73991-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:57.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-31"}]: dispatch 2024-06-18T13:07:57.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:57.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-31"}]': finished 2024-06-18T13:07:58.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:57 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:07:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:07:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi167-73991-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi167-73991-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:57 smithi179 ceph-mon[33312]: osdmap e250: 8 total, 8 up, 8 in 2024-06-18T13:07:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-31"}]: dispatch 2024-06-18T13:07:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi167-73991-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:07:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-31"}]: dispatch 2024-06-18T13:07:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-31"}]': finished 2024-06-18T13:07:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi167-73976-39"}]': finished 2024-06-18T13:07:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-31", "mode": "writeback"}]: dispatch 2024-06-18T13:07:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: osdmap e251: 8 total, 8 up, 8 in 2024-06-18T13:07:58.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:58.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-31", "mode": "writeback"}]: dispatch 2024-06-18T13:07:58.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: pgmap v329: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:58.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:58.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi167-73991-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi167-73991-49"}]': finished 2024-06-18T13:07:58.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi167-73976-39"}]': finished 2024-06-18T13:07:58.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-31", "mode": "writeback"}]': finished 2024-06-18T13:07:58.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: osdmap e252: 8 total, 8 up, 8 in 2024-06-18T13:07:58.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:58.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:58.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:58.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:58.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi167-73976-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:58.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi167-73976-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:58.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:58.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:58.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi167-73976-39"}]': finished 2024-06-18T13:07:58.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-31", "mode": "writeback"}]: dispatch 2024-06-18T13:07:58.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: osdmap e251: 8 total, 8 up, 8 in 2024-06-18T13:07:58.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:58.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-31", "mode": "writeback"}]: dispatch 2024-06-18T13:07:58.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: pgmap v329: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:58.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:58.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi167-73991-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi167-73991-49"}]': finished 2024-06-18T13:07:58.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi167-73976-39"}]': finished 2024-06-18T13:07:58.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-31", "mode": "writeback"}]': finished 2024-06-18T13:07:58.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: osdmap e252: 8 total, 8 up, 8 in 2024-06-18T13:07:58.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:58.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:58.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:58.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:58.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi167-73976-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:58.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi167-73976-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:58.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:58.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:59.139 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:07:59.140 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi167-73976-39"}]': finished 2024-06-18T13:07:59.140 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-31", "mode": "writeback"}]: dispatch 2024-06-18T13:07:59.140 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: osdmap e251: 8 total, 8 up, 8 in 2024-06-18T13:07:59.140 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi167-73976-39"}]: dispatch 2024-06-18T13:07:59.140 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-31", "mode": "writeback"}]: dispatch 2024-06-18T13:07:59.141 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: pgmap v329: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:07:59.141 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:07:59.141 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi167-73991-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi167-73991-49"}]': finished 2024-06-18T13:07:59.141 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/981373460' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi167-73976-39"}]': finished 2024-06-18T13:07:59.141 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-31", "mode": "writeback"}]': finished 2024-06-18T13:07:59.142 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: osdmap e252: 8 total, 8 up, 8 in 2024-06-18T13:07:59.142 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:59.142 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:59.142 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:59.142 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:59.142 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi167-73976-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:59.143 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi167-73976-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:07:59.143 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:59.143 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:07:59.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi167-73976-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[29496]: osdmap e253: 8 total, 8 up, 8 in 2024-06-18T13:07:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi167-73976-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31"}]: dispatch 2024-06-18T13:07:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi167-73976-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31"}]: dispatch 2024-06-18T13:07:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:07:59.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi167-73976-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:07:59.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:07:59.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[37701]: osdmap e253: 8 total, 8 up, 8 in 2024-06-18T13:07:59.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi167-73976-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:59.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31"}]: dispatch 2024-06-18T13:07:59.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi167-73976-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:07:59.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:07:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31"}]: dispatch 2024-06-18T13:08:00.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:59 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi167-73976-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:59 smithi179 ceph-mon[33312]: osdmap e253: 8 total, 8 up, 8 in 2024-06-18T13:08:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:59 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi167-73976-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:08:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:59 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31"}]: dispatch 2024-06-18T13:08:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi167-73976-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:08:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:07:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31"}]: dispatch 2024-06-18T13:08:00.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:00 smithi167 ceph-mon[29496]: pgmap v332: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-18T13:08:00.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:00 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31"}]': finished 2024-06-18T13:08:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:00 smithi167 ceph-mon[37701]: pgmap v332: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-18T13:08:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:00 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31"}]': finished 2024-06-18T13:08:01.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:00 smithi179 ceph-mon[33312]: pgmap v332: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-18T13:08:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:00 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-31"}]': finished 2024-06-18T13:08:01.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[37701]: osdmap e254: 8 total, 8 up, 8 in 2024-06-18T13:08:01.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:08:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:08:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi167-73976-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi167-73976-40"}]': finished 2024-06-18T13:08:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]': finished 2024-06-18T13:08:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[37701]: osdmap e255: 8 total, 8 up, 8 in 2024-06-18T13:08:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:08:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:08:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[29496]: osdmap e254: 8 total, 8 up, 8 in 2024-06-18T13:08:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:08:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:08:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi167-73976-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi167-73976-40"}]': finished 2024-06-18T13:08:01.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]': finished 2024-06-18T13:08:01.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[29496]: osdmap e255: 8 total, 8 up, 8 in 2024-06-18T13:08:01.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:08:01.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:08:02.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:01 smithi179 ceph-mon[33312]: osdmap e254: 8 total, 8 up, 8 in 2024-06-18T13:08:02.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:08:02.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:08:02.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi167-73976-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi167-73976-40"}]': finished 2024-06-18T13:08:02.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi167-73991-49"}]': finished 2024-06-18T13:08:02.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:01 smithi179 ceph-mon[33312]: osdmap e255: 8 total, 8 up, 8 in 2024-06-18T13:08:02.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4256466219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:08:02.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]: dispatch 2024-06-18T13:08:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[37701]: pgmap v335: 268 pgs: 8 unknown, 260 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:08:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]': finished 2024-06-18T13:08:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[37701]: osdmap e256: 8 total, 8 up, 8 in 2024-06-18T13:08:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:02.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:02.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:02.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi167-73991-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:02.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[29496]: pgmap v335: 268 pgs: 8 unknown, 260 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:08:02.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]': finished 2024-06-18T13:08:02.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[29496]: osdmap e256: 8 total, 8 up, 8 in 2024-06-18T13:08:02.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:02.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:02.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:02.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:02.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi167-73991-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:03.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:02 smithi179 ceph-mon[33312]: pgmap v335: 268 pgs: 8 unknown, 260 active+clean; 4.6 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:08:03.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi167-73991-49"}]': finished 2024-06-18T13:08:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:02 smithi179 ceph-mon[33312]: osdmap e256: 8 total, 8 up, 8 in 2024-06-18T13:08:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi167-73991-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:04.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi167-73991-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[37701]: osdmap e257: 8 total, 8 up, 8 in 2024-06-18T13:08:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi167-73991-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:08:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:08:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[37701]: pgmap v338: 292 pgs: 292 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-18T13:08:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi167-73991-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[29496]: osdmap e257: 8 total, 8 up, 8 in 2024-06-18T13:08:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi167-73991-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:08:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:08:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:04 smithi167 ceph-mon[29496]: pgmap v338: 292 pgs: 292 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-18T13:08:04.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi167-73991-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:04 smithi179 ceph-mon[33312]: osdmap e257: 8 total, 8 up, 8 in 2024-06-18T13:08:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi167-73991-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:08:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:08:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:04 smithi179 ceph-mon[33312]: pgmap v338: 292 pgs: 292 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-18T13:08:05.436 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:08:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:08:05.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]': finished 2024-06-18T13:08:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:08:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-33"}]: dispatch 2024-06-18T13:08:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[37701]: osdmap e258: 8 total, 8 up, 8 in 2024-06-18T13:08:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:08:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-33"}]: dispatch 2024-06-18T13:08:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]': finished 2024-06-18T13:08:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:08:05.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-33"}]: dispatch 2024-06-18T13:08:05.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[29496]: osdmap e258: 8 total, 8 up, 8 in 2024-06-18T13:08:05.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:08:05.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-33"}]: dispatch 2024-06-18T13:08:05.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:05 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi167-73976-40"}]': finished 2024-06-18T13:08:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3351956763' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:08:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-33"}]: dispatch 2024-06-18T13:08:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:05 smithi179 ceph-mon[33312]: osdmap e258: 8 total, 8 up, 8 in 2024-06-18T13:08:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]: dispatch 2024-06-18T13:08:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-33"}]: dispatch 2024-06-18T13:08:06.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi167-73991-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi167-73991-50"}]': finished 2024-06-18T13:08:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]': finished 2024-06-18T13:08:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-33"}]': finished 2024-06-18T13:08:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-33", "mode": "writeback"}]: dispatch 2024-06-18T13:08:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[37701]: osdmap e259: 8 total, 8 up, 8 in 2024-06-18T13:08:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-33", "mode": "writeback"}]: dispatch 2024-06-18T13:08:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi167-73976-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[37701]: pgmap v341: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:06.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi167-73991-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi167-73991-50"}]': finished 2024-06-18T13:08:06.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]': finished 2024-06-18T13:08:06.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-33"}]': finished 2024-06-18T13:08:06.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-33", "mode": "writeback"}]: dispatch 2024-06-18T13:08:06.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[29496]: osdmap e259: 8 total, 8 up, 8 in 2024-06-18T13:08:06.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-33", "mode": "writeback"}]: dispatch 2024-06-18T13:08:06.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:06.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:06.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi167-73976-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:06.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:06 smithi167 ceph-mon[29496]: pgmap v341: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:06.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi167-73991-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi167-73991-50"}]': finished 2024-06-18T13:08:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi167-73976-40"}]': finished 2024-06-18T13:08:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-33"}]': finished 2024-06-18T13:08:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-33", "mode": "writeback"}]: dispatch 2024-06-18T13:08:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:06 smithi179 ceph-mon[33312]: osdmap e259: 8 total, 8 up, 8 in 2024-06-18T13:08:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-33", "mode": "writeback"}]: dispatch 2024-06-18T13:08:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi167-73976-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:06 smithi179 ceph-mon[33312]: pgmap v341: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:07.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:07 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:08:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-33", "mode": "writeback"}]': finished 2024-06-18T13:08:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi167-73976-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:07 smithi167 ceph-mon[37701]: osdmap e260: 8 total, 8 up, 8 in 2024-06-18T13:08:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi167-73976-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:07 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:08:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-33", "mode": "writeback"}]': finished 2024-06-18T13:08:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi167-73976-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:07 smithi167 ceph-mon[29496]: osdmap e260: 8 total, 8 up, 8 in 2024-06-18T13:08:07.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi167-73976-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:07.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:07.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:08.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:07 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:08:08.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-33", "mode": "writeback"}]': finished 2024-06-18T13:08:08.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi167-73976-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:08.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:07 smithi179 ceph-mon[33312]: osdmap e260: 8 total, 8 up, 8 in 2024-06-18T13:08:08.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi167-73976-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:08.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:08.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33"}]: dispatch 2024-06-18T13:08:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: osdmap e261: 8 total, 8 up, 8 in 2024-06-18T13:08:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33"}]: dispatch 2024-06-18T13:08:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:08.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: pgmap v344: 292 pgs: 292 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:08.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.5", "id": [3, 1]}]: dispatch 2024-06-18T13:08:08.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.9", "id": [3, 1]}]: dispatch 2024-06-18T13:08:08.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]: dispatch 2024-06-18T13:08:08.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.1b", "id": [2, 5]}]: dispatch 2024-06-18T13:08:08.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "249.e", "id": [7, 3]}]: dispatch 2024-06-18T13:08:08.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "249.f", "id": [7, 3]}]: dispatch 2024-06-18T13:08:08.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:08.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi167-73976-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi167-73976-41"}]': finished 2024-06-18T13:08:08.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33"}]': finished 2024-06-18T13:08:08.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi167-73991-50"}]': finished 2024-06-18T13:08:08.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.5", "id": [3, 1]}]': finished 2024-06-18T13:08:08.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.9", "id": [3, 1]}]': finished 2024-06-18T13:08:08.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]': finished 2024-06-18T13:08:08.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.1b", "id": [2, 5]}]': finished 2024-06-18T13:08:08.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:08.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33"}]: dispatch 2024-06-18T13:08:08.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: osdmap e261: 8 total, 8 up, 8 in 2024-06-18T13:08:08.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33"}]: dispatch 2024-06-18T13:08:08.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:08.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: pgmap v344: 292 pgs: 292 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:08.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.5", "id": [3, 1]}]: dispatch 2024-06-18T13:08:08.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.9", "id": [3, 1]}]: dispatch 2024-06-18T13:08:08.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]: dispatch 2024-06-18T13:08:08.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.1b", "id": [2, 5]}]: dispatch 2024-06-18T13:08:08.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "249.e", "id": [7, 3]}]: dispatch 2024-06-18T13:08:08.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "249.f", "id": [7, 3]}]: dispatch 2024-06-18T13:08:08.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:08.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi167-73976-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi167-73976-41"}]': finished 2024-06-18T13:08:08.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33"}]': finished 2024-06-18T13:08:08.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi167-73991-50"}]': finished 2024-06-18T13:08:08.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.5", "id": [3, 1]}]': finished 2024-06-18T13:08:08.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.9", "id": [3, 1]}]': finished 2024-06-18T13:08:08.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]': finished 2024-06-18T13:08:08.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.1b", "id": [2, 5]}]': finished 2024-06-18T13:08:08.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "249.e", "id": [7, 3]}]': finished 2024-06-18T13:08:08.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "249.f", "id": [7, 3]}]': finished 2024-06-18T13:08:08.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: osdmap e262: 8 total, 8 up, 8 in 2024-06-18T13:08:08.981 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:08.981 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "249.e", "id": [7, 3]}]': finished 2024-06-18T13:08:08.981 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "249.f", "id": [7, 3]}]': finished 2024-06-18T13:08:08.981 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: osdmap e262: 8 total, 8 up, 8 in 2024-06-18T13:08:08.981 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:09.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:08:09.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:09.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33"}]: dispatch 2024-06-18T13:08:09.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: osdmap e261: 8 total, 8 up, 8 in 2024-06-18T13:08:09.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33"}]: dispatch 2024-06-18T13:08:09.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:09.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: pgmap v344: 292 pgs: 292 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:09.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.5", "id": [3, 1]}]: dispatch 2024-06-18T13:08:09.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.9", "id": [3, 1]}]: dispatch 2024-06-18T13:08:09.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]: dispatch 2024-06-18T13:08:09.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.1b", "id": [2, 5]}]: dispatch 2024-06-18T13:08:09.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "249.e", "id": [7, 3]}]: dispatch 2024-06-18T13:08:09.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "249.f", "id": [7, 3]}]: dispatch 2024-06-18T13:08:09.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:09.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi167-73976-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi167-73976-41"}]': finished 2024-06-18T13:08:09.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-33"}]': finished 2024-06-18T13:08:09.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi167-73991-50"}]': finished 2024-06-18T13:08:09.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.5", "id": [3, 1]}]': finished 2024-06-18T13:08:09.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.9", "id": [3, 1]}]': finished 2024-06-18T13:08:09.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]': finished 2024-06-18T13:08:09.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.1b", "id": [2, 5]}]': finished 2024-06-18T13:08:09.139 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "249.e", "id": [7, 3]}]': finished 2024-06-18T13:08:09.139 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "249.f", "id": [7, 3]}]': finished 2024-06-18T13:08:09.139 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: osdmap e262: 8 total, 8 up, 8 in 2024-06-18T13:08:09.139 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi167-73991-50"}]: dispatch 2024-06-18T13:08:10.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:09 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:10.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:08:10.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi167-73991-50"}]': finished 2024-06-18T13:08:10.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:09 smithi179 ceph-mon[33312]: osdmap e263: 8 total, 8 up, 8 in 2024-06-18T13:08:10.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:09 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:10.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:08:10.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi167-73991-50"}]': finished 2024-06-18T13:08:10.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:09 smithi167 ceph-mon[37701]: osdmap e263: 8 total, 8 up, 8 in 2024-06-18T13:08:10.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:09 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:10.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:08:10.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1866047483' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi167-73991-50"}]': finished 2024-06-18T13:08:10.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:09 smithi167 ceph-mon[29496]: osdmap e263: 8 total, 8 up, 8 in 2024-06-18T13:08:11.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:10 smithi167 ceph-mon[37701]: pgmap v347: 300 pgs: 8 creating+peering, 6 peering, 286 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-06-18T13:08:11.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:10 smithi167 ceph-mon[37701]: osdmap e264: 8 total, 8 up, 8 in 2024-06-18T13:08:11.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3051546465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:11.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:10 smithi167 ceph-mon[29496]: pgmap v347: 300 pgs: 8 creating+peering, 6 peering, 286 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-06-18T13:08:11.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:10 smithi167 ceph-mon[29496]: osdmap e264: 8 total, 8 up, 8 in 2024-06-18T13:08:11.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3051546465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:11.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:10 smithi179 ceph-mon[33312]: pgmap v347: 300 pgs: 8 creating+peering, 6 peering, 286 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-06-18T13:08:11.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:10 smithi179 ceph-mon[33312]: osdmap e264: 8 total, 8 up, 8 in 2024-06-18T13:08:11.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3051546465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:12.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[37701]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-06-18T13:08:12.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:12.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:12.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3051546465' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:12.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi167-73976-41"}]': finished 2024-06-18T13:08:12.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[37701]: osdmap e265: 8 total, 8 up, 8 in 2024-06-18T13:08:12.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:12.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:12.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:12.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[29496]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-06-18T13:08:12.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:12.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:12.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3051546465' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:12.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi167-73976-41"}]': finished 2024-06-18T13:08:12.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[29496]: osdmap e265: 8 total, 8 up, 8 in 2024-06-18T13:08:12.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:12.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:12.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:12.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:11 smithi179 ceph-mon[33312]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-06-18T13:08:12.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:11 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:12.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:12.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3051546465' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi167-73991-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:12.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi167-73976-41"}]': finished 2024-06-18T13:08:12.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:11 smithi179 ceph-mon[33312]: osdmap e265: 8 total, 8 up, 8 in 2024-06-18T13:08:12.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi167-73976-41"}]: dispatch 2024-06-18T13:08:12.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:12.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:13.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: pgmap v350: 324 pgs: 96 unknown, 1 peering, 227 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:08:13.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi167-73976-41"}]': finished 2024-06-18T13:08:13.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:13.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: osdmap e266: 8 total, 8 up, 8 in 2024-06-18T13:08:13.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:13.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:13.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:13.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:13.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:13.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:13.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi167-73976-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:13.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:13.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:13.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:13.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:13.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: pgmap v350: 324 pgs: 96 unknown, 1 peering, 227 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:08:13.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi167-73976-41"}]': finished 2024-06-18T13:08:13.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:13.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: osdmap e266: 8 total, 8 up, 8 in 2024-06-18T13:08:13.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:13.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:13.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:13.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:13.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:13.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:13.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi167-73976-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:13.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:13.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:13.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:13.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: pgmap v350: 324 pgs: 96 unknown, 1 peering, 227 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:08:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3285614431' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi167-73976-41"}]': finished 2024-06-18T13:08:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: osdmap e266: 8 total, 8 up, 8 in 2024-06-18T13:08:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:13.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:13.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi167-73976-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:13.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:13.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:13.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:13.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[29496]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-06-18T13:08:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi167-73976-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[29496]: osdmap e267: 8 total, 8 up, 8 in 2024-06-18T13:08:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi167-73991-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi167-73976-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi167-73991-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:14.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[37701]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-06-18T13:08:14.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:14.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi167-73976-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:14.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:14.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[37701]: osdmap e267: 8 total, 8 up, 8 in 2024-06-18T13:08:14.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:14.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi167-73991-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:14.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi167-73976-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:14.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:14.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi167-73991-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:14.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:13 smithi179 ceph-mon[33312]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-06-18T13:08:14.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:14.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi167-73976-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:14.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:14.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:13 smithi179 ceph-mon[33312]: osdmap e267: 8 total, 8 up, 8 in 2024-06-18T13:08:14.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:14.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi167-73991-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:14.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi167-73976-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:14.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:14.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi167-73991-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:15.226 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:08:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:08:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:08:15.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:14 smithi167 ceph-mon[37701]: pgmap v353: 260 pgs: 260 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 10 B/s, 3 objects/s recovering 2024-06-18T13:08:15.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]': finished 2024-06-18T13:08:15.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-35", "mode": "writeback"}]: dispatch 2024-06-18T13:08:15.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:14 smithi167 ceph-mon[37701]: osdmap e268: 8 total, 8 up, 8 in 2024-06-18T13:08:15.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-35", "mode": "writeback"}]: dispatch 2024-06-18T13:08:15.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:14 smithi167 ceph-mon[29496]: pgmap v353: 260 pgs: 260 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 10 B/s, 3 objects/s recovering 2024-06-18T13:08:15.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]': finished 2024-06-18T13:08:15.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-35", "mode": "writeback"}]: dispatch 2024-06-18T13:08:15.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:14 smithi167 ceph-mon[29496]: osdmap e268: 8 total, 8 up, 8 in 2024-06-18T13:08:15.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-35", "mode": "writeback"}]: dispatch 2024-06-18T13:08:15.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:14 smithi179 ceph-mon[33312]: pgmap v353: 260 pgs: 260 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 10 B/s, 3 objects/s recovering 2024-06-18T13:08:15.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]': finished 2024-06-18T13:08:15.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-35", "mode": "writeback"}]: dispatch 2024-06-18T13:08:15.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:14 smithi179 ceph-mon[33312]: osdmap e268: 8 total, 8 up, 8 in 2024-06-18T13:08:15.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-35", "mode": "writeback"}]: dispatch 2024-06-18T13:08:16.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:15 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:08:16.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi167-73976-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi167-73976-42"}]': finished 2024-06-18T13:08:16.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi167-73991-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]': finished 2024-06-18T13:08:16.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-35", "mode": "writeback"}]': finished 2024-06-18T13:08:16.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:15 smithi167 ceph-mon[29496]: osdmap e269: 8 total, 8 up, 8 in 2024-06-18T13:08:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:15 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:08:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi167-73976-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi167-73976-42"}]': finished 2024-06-18T13:08:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi167-73991-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]': finished 2024-06-18T13:08:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-35", "mode": "writeback"}]': finished 2024-06-18T13:08:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:15 smithi167 ceph-mon[37701]: osdmap e269: 8 total, 8 up, 8 in 2024-06-18T13:08:16.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:15 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:08:16.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi167-73976-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi167-73976-42"}]': finished 2024-06-18T13:08:16.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi167-73991-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]': finished 2024-06-18T13:08:16.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-35", "mode": "writeback"}]': finished 2024-06-18T13:08:16.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:15 smithi179 ceph-mon[33312]: osdmap e269: 8 total, 8 up, 8 in 2024-06-18T13:08:17.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:16 smithi167 ceph-mon[37701]: pgmap v356: 308 pgs: 48 unknown, 260 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 10 B/s, 3 objects/s recovering 2024-06-18T13:08:17.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:16 smithi167 ceph-mon[37701]: osdmap e270: 8 total, 8 up, 8 in 2024-06-18T13:08:17.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:16 smithi167 ceph-mon[29496]: pgmap v356: 308 pgs: 48 unknown, 260 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 10 B/s, 3 objects/s recovering 2024-06-18T13:08:17.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:16 smithi167 ceph-mon[29496]: osdmap e270: 8 total, 8 up, 8 in 2024-06-18T13:08:17.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:16 smithi179 ceph-mon[33312]: pgmap v356: 308 pgs: 48 unknown, 260 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 10 B/s, 3 objects/s recovering 2024-06-18T13:08:17.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:16 smithi179 ceph-mon[33312]: osdmap e270: 8 total, 8 up, 8 in 2024-06-18T13:08:18.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:17 smithi179 ceph-mon[33312]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:18.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:17 smithi179 ceph-mon[33312]: osdmap e271: 8 total, 8 up, 8 in 2024-06-18T13:08:18.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:18.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:18.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:18.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:18.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:18.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:17 smithi167 ceph-mon[37701]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:18.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:17 smithi167 ceph-mon[37701]: osdmap e271: 8 total, 8 up, 8 in 2024-06-18T13:08:18.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:18.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:18.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:18.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:18.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:18.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:17 smithi167 ceph-mon[29496]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:18.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:17 smithi167 ceph-mon[29496]: osdmap e271: 8 total, 8 up, 8 in 2024-06-18T13:08:18.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:18.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:18.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:18.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:18.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:18.985 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:08:18 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:08:19.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:18 smithi179 ceph-mon[33312]: pgmap v359: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:19.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]': finished 2024-06-18T13:08:19.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi167-73976-42"}]': finished 2024-06-18T13:08:19.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:19.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:18 smithi179 ceph-mon[33312]: osdmap e272: 8 total, 8 up, 8 in 2024-06-18T13:08:19.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:19.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:19.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:19.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:19.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:19.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[37701]: pgmap v359: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:19.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]': finished 2024-06-18T13:08:19.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi167-73976-42"}]': finished 2024-06-18T13:08:19.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:19.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[37701]: osdmap e272: 8 total, 8 up, 8 in 2024-06-18T13:08:19.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:19.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:19.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:19.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:19.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:19.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[29496]: pgmap v359: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:19.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi167-73991-52"}]': finished 2024-06-18T13:08:19.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi167-73976-42"}]': finished 2024-06-18T13:08:19.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:19.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[29496]: osdmap e272: 8 total, 8 up, 8 in 2024-06-18T13:08:19.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi167-73976-42"}]: dispatch 2024-06-18T13:08:19.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/882995649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:19.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]: dispatch 2024-06-18T13:08:19.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:19.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:19.496 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7021 ms) 2024-06-18T13:08:19.496 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-06-18T13:08:19.496 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6937 ms) 2024-06-18T13:08:19.496 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-06-18T13:08:19.496 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7001 ms) 2024-06-18T13:08:19.496 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-06-18T13:08:19.496 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7239 ms) 2024-06-18T13:08:19.496 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-06-18T13:08:19.496 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (6771 ms) 2024-06-18T13:08:19.496 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-06-18T13:08:19.497 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7036 ms) 2024-06-18T13:08:19.497 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-06-18T13:08:19.497 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7036 ms) 2024-06-18T13:08:19.497 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-06-18T13:08:19.497 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7020 ms) 2024-06-18T13:08:19.497 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-06-18T13:08:19.497 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6898 ms) 2024-06-18T13:08:19.497 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-06-18T13:08:19.497 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7018 ms) 2024-06-18T13:08:19.497 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-06-18T13:08:19.497 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6985 ms) 2024-06-18T13:08:19.497 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-06-18T13:08:19.497 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (7054 ms) 2024-06-18T13:08:19.497 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-06-18T13:08:19.498 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6956 ms) 2024-06-18T13:08:19.498 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-06-18T13:08:19.498 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7030 ms) 2024-06-18T13:08:19.498 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-06-18T13:08:19.498 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7018 ms) 2024-06-18T13:08:19.498 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-06-18T13:08:19.498 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7017 ms) 2024-06-18T13:08:19.498 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112037 ms total) 2024-06-18T13:08:19.498 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: 2024-06-18T13:08:19.498 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [----------] Global test environment tear-down 2024-06-18T13:08:19.498 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (205460 ms total) 2024-06-18T13:08:19.498 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio: [ PASSED ] 42 tests. 2024-06-18T13:08:20.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:19 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi167-73976-42"}]': finished 2024-06-18T13:08:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]': finished 2024-06-18T13:08:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]': finished 2024-06-18T13:08:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:19 smithi179 ceph-mon[33312]: osdmap e273: 8 total, 8 up, 8 in 2024-06-18T13:08:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:20.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi167-73976-42"}]': finished 2024-06-18T13:08:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]': finished 2024-06-18T13:08:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]': finished 2024-06-18T13:08:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[37701]: osdmap e273: 8 total, 8 up, 8 in 2024-06-18T13:08:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:20.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:20.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:20.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:20.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:20.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:20.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/936240863' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi167-73976-42"}]': finished 2024-06-18T13:08:20.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi167-73991-52"}]': finished 2024-06-18T13:08:20.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-35"}]': finished 2024-06-18T13:08:20.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[29496]: osdmap e273: 8 total, 8 up, 8 in 2024-06-18T13:08:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:20.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:21.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:20 smithi179 ceph-mon[33312]: pgmap v362: 260 pgs: 260 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:08:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:20 smithi179 ceph-mon[33312]: osdmap e274: 8 total, 8 up, 8 in 2024-06-18T13:08:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi167-73991-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi167-73991-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:21.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:20 smithi167 ceph-mon[37701]: pgmap v362: 260 pgs: 260 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:08:21.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:21.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:20 smithi167 ceph-mon[37701]: osdmap e274: 8 total, 8 up, 8 in 2024-06-18T13:08:21.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi167-73991-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:21.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi167-73991-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:21.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:21.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:21.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:20 smithi167 ceph-mon[29496]: pgmap v362: 260 pgs: 260 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:08:21.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:21.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:20 smithi167 ceph-mon[29496]: osdmap e274: 8 total, 8 up, 8 in 2024-06-18T13:08:21.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi167-73991-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:21.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi167-73991-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:21.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:21.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:22.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:22 smithi179 ceph-mon[33312]: osdmap e275: 8 total, 8 up, 8 in 2024-06-18T13:08:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:22 smithi179 ceph-mon[33312]: pgmap v365: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:08:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:22 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:22 smithi167 ceph-mon[37701]: osdmap e275: 8 total, 8 up, 8 in 2024-06-18T13:08:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:22 smithi167 ceph-mon[37701]: pgmap v365: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:08:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:22 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:22 smithi167 ceph-mon[29496]: osdmap e275: 8 total, 8 up, 8 in 2024-06-18T13:08:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35"}]: dispatch 2024-06-18T13:08:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:22 smithi167 ceph-mon[29496]: pgmap v365: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:08:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:22 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:24.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:23 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:24.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi167-73991-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi167-73991-53"}]': finished 2024-06-18T13:08:24.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35"}]': finished 2024-06-18T13:08:24.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:23 smithi179 ceph-mon[33312]: osdmap e276: 8 total, 8 up, 8 in 2024-06-18T13:08:24.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:23 smithi179 ceph-mon[33312]: osdmap e277: 8 total, 8 up, 8 in 2024-06-18T13:08:24.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:23 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:24.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi167-73991-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi167-73991-53"}]': finished 2024-06-18T13:08:24.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35"}]': finished 2024-06-18T13:08:24.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:23 smithi167 ceph-mon[37701]: osdmap e276: 8 total, 8 up, 8 in 2024-06-18T13:08:24.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:23 smithi167 ceph-mon[37701]: osdmap e277: 8 total, 8 up, 8 in 2024-06-18T13:08:24.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:23 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:24.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi167-73991-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi167-73991-53"}]': finished 2024-06-18T13:08:24.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-35"}]': finished 2024-06-18T13:08:24.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:23 smithi167 ceph-mon[29496]: osdmap e276: 8 total, 8 up, 8 in 2024-06-18T13:08:24.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:23 smithi167 ceph-mon[29496]: osdmap e277: 8 total, 8 up, 8 in 2024-06-18T13:08:25.031 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:24 smithi167 ceph-mon[37701]: pgmap v368: 268 pgs: 33 creating+peering, 5 creating+activating, 2 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:08:25.031 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:24 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:08:25.031 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:24 smithi167 ceph-mon[37701]: osdmap e278: 8 total, 8 up, 8 in 2024-06-18T13:08:25.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:25.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:25.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:25.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:25.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:24 smithi167 ceph-mon[29496]: pgmap v368: 268 pgs: 33 creating+peering, 5 creating+activating, 2 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:08:25.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:24 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:08:25.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:24 smithi167 ceph-mon[29496]: osdmap e278: 8 total, 8 up, 8 in 2024-06-18T13:08:25.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:25.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:25.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:25.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:25.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:24 smithi179 ceph-mon[33312]: pgmap v368: 268 pgs: 33 creating+peering, 5 creating+activating, 2 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:08:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:24 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:08:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:24 smithi179 ceph-mon[33312]: osdmap e278: 8 total, 8 up, 8 in 2024-06-18T13:08:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:25.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:08:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:08:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:08:26.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]': finished 2024-06-18T13:08:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:26 smithi179 ceph-mon[33312]: osdmap e279: 8 total, 8 up, 8 in 2024-06-18T13:08:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:26 smithi179 ceph-mon[33312]: pgmap v371: 260 pgs: 32 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:08:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]': finished 2024-06-18T13:08:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[37701]: osdmap e279: 8 total, 8 up, 8 in 2024-06-18T13:08:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[37701]: pgmap v371: 260 pgs: 32 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:08:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi167-73991-53"}]': finished 2024-06-18T13:08:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[29496]: osdmap e279: 8 total, 8 up, 8 in 2024-06-18T13:08:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1858972201' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]: dispatch 2024-06-18T13:08:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:26 smithi167 ceph-mon[29496]: pgmap v371: 260 pgs: 32 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:08:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]': finished 2024-06-18T13:08:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: osdmap e280: 8 total, 8 up, 8 in 2024-06-18T13:08:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-37"}]: dispatch 2024-06-18T13:08:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-37"}]: dispatch 2024-06-18T13:08:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi167-73991-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:27.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi167-73991-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:27.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-37"}]': finished 2024-06-18T13:08:27.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi167-73991-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:27.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: osdmap e281: 8 total, 8 up, 8 in 2024-06-18T13:08:27.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi167-73991-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi167-73991-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-37", "mode": "writeback"}]: dispatch 2024-06-18T13:08:27.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-37", "mode": "writeback"}]: dispatch 2024-06-18T13:08:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]': finished 2024-06-18T13:08:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: osdmap e280: 8 total, 8 up, 8 in 2024-06-18T13:08:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-37"}]: dispatch 2024-06-18T13:08:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-37"}]: dispatch 2024-06-18T13:08:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi167-73991-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:27.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi167-73991-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:27.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-37"}]': finished 2024-06-18T13:08:27.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi167-73991-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:27.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: osdmap e281: 8 total, 8 up, 8 in 2024-06-18T13:08:27.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi167-73991-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi167-73991-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-37", "mode": "writeback"}]: dispatch 2024-06-18T13:08:27.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-37", "mode": "writeback"}]: dispatch 2024-06-18T13:08:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi167-73991-53"}]': finished 2024-06-18T13:08:27.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:27.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: osdmap e280: 8 total, 8 up, 8 in 2024-06-18T13:08:27.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-37"}]: dispatch 2024-06-18T13:08:27.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-37"}]: dispatch 2024-06-18T13:08:27.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi167-73991-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:27.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi167-73991-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:27.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-37"}]': finished 2024-06-18T13:08:27.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi167-73991-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:27.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: osdmap e281: 8 total, 8 up, 8 in 2024-06-18T13:08:27.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi167-73991-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi167-73991-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:27.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-37", "mode": "writeback"}]: dispatch 2024-06-18T13:08:27.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-37", "mode": "writeback"}]: dispatch 2024-06-18T13:08:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:28 smithi167 ceph-mon[37701]: pgmap v374: 292 pgs: 64 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:28 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:08:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-37", "mode": "writeback"}]': finished 2024-06-18T13:08:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:28 smithi167 ceph-mon[29496]: pgmap v374: 292 pgs: 64 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:28 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:08:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-37", "mode": "writeback"}]': finished 2024-06-18T13:08:29.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:08:28 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:08:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:28 smithi179 ceph-mon[33312]: pgmap v374: 292 pgs: 64 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:28 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:08:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-37", "mode": "writeback"}]': finished 2024-06-18T13:08:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[37701]: osdmap e282: 8 total, 8 up, 8 in 2024-06-18T13:08:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi167-73991-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi167-73991-54"}]': finished 2024-06-18T13:08:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[37701]: osdmap e283: 8 total, 8 up, 8 in 2024-06-18T13:08:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37"}]: dispatch 2024-06-18T13:08:29.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37"}]: dispatch 2024-06-18T13:08:29.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:29.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[29496]: osdmap e282: 8 total, 8 up, 8 in 2024-06-18T13:08:29.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:29.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:29.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:29.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi167-73991-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi167-73991-54"}]': finished 2024-06-18T13:08:29.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:29.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[29496]: osdmap e283: 8 total, 8 up, 8 in 2024-06-18T13:08:29.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37"}]: dispatch 2024-06-18T13:08:29.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37"}]: dispatch 2024-06-18T13:08:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:29 smithi179 ceph-mon[33312]: osdmap e282: 8 total, 8 up, 8 in 2024-06-18T13:08:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:29 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:29 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi167-73991-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi167-73991-54"}]': finished 2024-06-18T13:08:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:29 smithi179 ceph-mon[33312]: osdmap e283: 8 total, 8 up, 8 in 2024-06-18T13:08:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37"}]: dispatch 2024-06-18T13:08:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37"}]: dispatch 2024-06-18T13:08:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:30 smithi167 ceph-mon[37701]: pgmap v377: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-06-18T13:08:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:30 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37"}]': finished 2024-06-18T13:08:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:30 smithi167 ceph-mon[37701]: osdmap e284: 8 total, 8 up, 8 in 2024-06-18T13:08:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:30 smithi167 ceph-mon[29496]: pgmap v377: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-06-18T13:08:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:30 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37"}]': finished 2024-06-18T13:08:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:30 smithi167 ceph-mon[29496]: osdmap e284: 8 total, 8 up, 8 in 2024-06-18T13:08:31.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:30 smithi179 ceph-mon[33312]: pgmap v377: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-06-18T13:08:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:30 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-37"}]': finished 2024-06-18T13:08:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:30 smithi179 ceph-mon[33312]: osdmap e284: 8 total, 8 up, 8 in 2024-06-18T13:08:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:32 smithi167 ceph-mon[37701]: osdmap e285: 8 total, 8 up, 8 in 2024-06-18T13:08:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:32 smithi167 ceph-mon[37701]: pgmap v380: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:08:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:32 smithi167 ceph-mon[29496]: osdmap e285: 8 total, 8 up, 8 in 2024-06-18T13:08:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:32 smithi167 ceph-mon[29496]: pgmap v380: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:08:32.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:32 smithi179 ceph-mon[33312]: osdmap e285: 8 total, 8 up, 8 in 2024-06-18T13:08:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:32 smithi179 ceph-mon[33312]: pgmap v380: 228 pgs: 228 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:08:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:33 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]': finished 2024-06-18T13:08:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:33 smithi167 ceph-mon[37701]: osdmap e286: 8 total, 8 up, 8 in 2024-06-18T13:08:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:33 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:33 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:33 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]': finished 2024-06-18T13:08:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:33 smithi167 ceph-mon[29496]: osdmap e286: 8 total, 8 up, 8 in 2024-06-18T13:08:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:33.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:33 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:33.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:33.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:33 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:33.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:33 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi167-73991-54"}]': finished 2024-06-18T13:08:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:33 smithi179 ceph-mon[33312]: osdmap e286: 8 total, 8 up, 8 in 2024-06-18T13:08:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4038604278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:33 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]: dispatch 2024-06-18T13:08:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:33 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]': finished 2024-06-18T13:08:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[37701]: osdmap e287: 8 total, 8 up, 8 in 2024-06-18T13:08:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi167-73991-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi167-73991-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:34.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[37701]: pgmap v383: 292 pgs: 292 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:34.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]': finished 2024-06-18T13:08:34.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:34.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[29496]: osdmap e287: 8 total, 8 up, 8 in 2024-06-18T13:08:34.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:34.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:34.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:34.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:34.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:34.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:34.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi167-73991-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:34.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi167-73991-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:34.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:34 smithi167 ceph-mon[29496]: pgmap v383: 292 pgs: 292 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:34.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi167-73991-54"}]': finished 2024-06-18T13:08:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:34 smithi179 ceph-mon[33312]: osdmap e287: 8 total, 8 up, 8 in 2024-06-18T13:08:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi167-73991-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi167-73991-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:34 smithi179 ceph-mon[33312]: pgmap v383: 292 pgs: 292 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:35.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:08:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:08:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi167-73991-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[29496]: osdmap e288: 8 total, 8 up, 8 in 2024-06-18T13:08:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-39"}]: dispatch 2024-06-18T13:08:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi167-73991-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-39"}]: dispatch 2024-06-18T13:08:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi167-73991-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:35.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:35.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:35.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi167-73991-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:35.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[37701]: osdmap e288: 8 total, 8 up, 8 in 2024-06-18T13:08:35.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-39"}]: dispatch 2024-06-18T13:08:35.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi167-73991-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:35.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-39"}]: dispatch 2024-06-18T13:08:35.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi167-73991-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:36.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:35 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi167-73991-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:35 smithi179 ceph-mon[33312]: osdmap e288: 8 total, 8 up, 8 in 2024-06-18T13:08:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-39"}]: dispatch 2024-06-18T13:08:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi167-73991-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-39"}]: dispatch 2024-06-18T13:08:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi167-73991-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:36.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-39"}]': finished 2024-06-18T13:08:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-39", "mode": "writeback"}]: dispatch 2024-06-18T13:08:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[29496]: osdmap e289: 8 total, 8 up, 8 in 2024-06-18T13:08:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-39", "mode": "writeback"}]: dispatch 2024-06-18T13:08:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[29496]: pgmap v386: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:08:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi167-73991-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi167-73991-55"}]': finished 2024-06-18T13:08:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-39", "mode": "writeback"}]': finished 2024-06-18T13:08:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[29496]: osdmap e290: 8 total, 8 up, 8 in 2024-06-18T13:08:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:36.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-39"}]': finished 2024-06-18T13:08:36.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-39", "mode": "writeback"}]: dispatch 2024-06-18T13:08:36.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[37701]: osdmap e289: 8 total, 8 up, 8 in 2024-06-18T13:08:36.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-39", "mode": "writeback"}]: dispatch 2024-06-18T13:08:36.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[37701]: pgmap v386: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:36.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:08:36.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi167-73991-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi167-73991-55"}]': finished 2024-06-18T13:08:36.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-39", "mode": "writeback"}]': finished 2024-06-18T13:08:36.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[37701]: osdmap e290: 8 total, 8 up, 8 in 2024-06-18T13:08:36.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:36.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:37.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-39"}]': finished 2024-06-18T13:08:37.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-39", "mode": "writeback"}]: dispatch 2024-06-18T13:08:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:36 smithi179 ceph-mon[33312]: osdmap e289: 8 total, 8 up, 8 in 2024-06-18T13:08:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-39", "mode": "writeback"}]: dispatch 2024-06-18T13:08:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:36 smithi179 ceph-mon[33312]: pgmap v386: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:36 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:08:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi167-73991-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi167-73991-55"}]': finished 2024-06-18T13:08:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-39", "mode": "writeback"}]': finished 2024-06-18T13:08:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:36 smithi179 ceph-mon[33312]: osdmap e290: 8 total, 8 up, 8 in 2024-06-18T13:08:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:38 smithi167 ceph-mon[37701]: osdmap e291: 8 total, 8 up, 8 in 2024-06-18T13:08:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:38 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39"}]: dispatch 2024-06-18T13:08:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39"}]: dispatch 2024-06-18T13:08:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:38 smithi167 ceph-mon[37701]: pgmap v389: 268 pgs: 8 unknown, 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:38.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:38.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:38 smithi167 ceph-mon[29496]: osdmap e291: 8 total, 8 up, 8 in 2024-06-18T13:08:38.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:38 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39"}]: dispatch 2024-06-18T13:08:38.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39"}]: dispatch 2024-06-18T13:08:38.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:38 smithi167 ceph-mon[29496]: pgmap v389: 268 pgs: 8 unknown, 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:38.731 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:08:38.731 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:38 smithi179 ceph-mon[33312]: osdmap e291: 8 total, 8 up, 8 in 2024-06-18T13:08:38.731 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:38 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39"}]: dispatch 2024-06-18T13:08:38.731 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39"}]: dispatch 2024-06-18T13:08:38.731 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:38 smithi179 ceph-mon[33312]: pgmap v389: 268 pgs: 8 unknown, 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:39.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:08:38 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:08:39.476 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-06-18T13:08:39.476 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-06-18T13:08:39.476 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-06-18T13:08:39.476 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: seed 74266 2024-06-18T13:08:39.476 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-06-18T13:08:39.476 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (292 ms) 2024-06-18T13:08:39.476 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-06-18T13:08:39.476 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (14956 ms) 2024-06-18T13:08:39.476 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-06-18T13:08:39.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (117 ms) 2024-06-18T13:08:39.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (15365 ms total) 2024-06-18T13:08:39.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: 2024-06-18T13:08:39.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-06-18T13:08:39.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-06-18T13:08:39.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (8517 ms) 2024-06-18T13:08:39.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-06-18T13:08:39.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (9616 ms) 2024-06-18T13:08:39.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-06-18T13:08:39.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (10107 ms) 2024-06-18T13:08:39.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-06-18T13:08:39.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: my_snaps [3] 2024-06-18T13:08:39.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: my_snaps [4,3] 2024-06-18T13:08:39.477 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: my_snaps [5,4,3] 2024-06-18T13:08:39.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-06-18T13:08:39.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: promoting some heads 2024-06-18T13:08:39.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: promoting from clones for snap 6 2024-06-18T13:08:39.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: promoting from clones for snap 5 2024-06-18T13:08:39.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: promoting from clones for snap 4 2024-06-18T13:08:39.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: promoting from clones for snap 3 2024-06-18T13:08:39.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: waiting for scrubs... 2024-06-18T13:08:39.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: done waiting 2024-06-18T13:08:39.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (45764 ms) 2024-06-18T13:08:39.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-06-18T13:08:39.478 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (10009 ms) 2024-06-18T13:08:39.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-06-18T13:08:39.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7998 ms) 2024-06-18T13:08:39.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-06-18T13:08:39.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8022 ms) 2024-06-18T13:08:39.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-06-18T13:08:39.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8276 ms) 2024-06-18T13:08:39.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-06-18T13:08:39.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (9799 ms) 2024-06-18T13:08:39.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-06-18T13:08:39.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8910 ms) 2024-06-18T13:08:39.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-06-18T13:08:39.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10026 ms) 2024-06-18T13:08:39.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-06-18T13:08:39.479 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (13283 ms) 2024-06-18T13:08:39.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-06-18T13:08:39.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7702 ms) 2024-06-18T13:08:39.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-06-18T13:08:39.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (9041 ms) 2024-06-18T13:08:39.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-06-18T13:08:39.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13037 ms) 2024-06-18T13:08:39.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-06-18T13:08:39.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7927 ms) 2024-06-18T13:08:39.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-06-18T13:08:39.480 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8046 ms) 2024-06-18T13:08:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39"}]': finished 2024-06-18T13:08:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: osdmap e292: 8 total, 8 up, 8 in 2024-06-18T13:08:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:08:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:08:39.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:08:39.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:08:39.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:39.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39"}]': finished 2024-06-18T13:08:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: osdmap e292: 8 total, 8 up, 8 in 2024-06-18T13:08:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:08:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:08:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:08:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:08:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:08:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-39"}]': finished 2024-06-18T13:08:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: osdmap e292: 8 total, 8 up, 8 in 2024-06-18T13:08:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:08:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:08:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:08:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:08:39.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:08:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]': finished 2024-06-18T13:08:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:40 smithi179 ceph-mon[33312]: osdmap e293: 8 total, 8 up, 8 in 2024-06-18T13:08:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:40 smithi179 ceph-mon[33312]: pgmap v392: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-06-18T13:08:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]': finished 2024-06-18T13:08:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:40 smithi179 ceph-mon[33312]: osdmap e294: 8 total, 8 up, 8 in 2024-06-18T13:08:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:40.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]': finished 2024-06-18T13:08:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[29496]: osdmap e293: 8 total, 8 up, 8 in 2024-06-18T13:08:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[29496]: pgmap v392: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-06-18T13:08:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]': finished 2024-06-18T13:08:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[29496]: osdmap e294: 8 total, 8 up, 8 in 2024-06-18T13:08:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi167-73991-55"}]': finished 2024-06-18T13:08:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[37701]: osdmap e293: 8 total, 8 up, 8 in 2024-06-18T13:08:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3995074001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]: dispatch 2024-06-18T13:08:40.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[37701]: pgmap v392: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-06-18T13:08:40.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi167-73991-55"}]': finished 2024-06-18T13:08:40.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[37701]: osdmap e294: 8 total, 8 up, 8 in 2024-06-18T13:08:40.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi167-73991-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi167-73991-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi167-73991-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:41 smithi179 ceph-mon[33312]: osdmap e295: 8 total, 8 up, 8 in 2024-06-18T13:08:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi167-73991-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:41.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi167-73991-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi167-73991-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:41.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi167-73991-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:41.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:41.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi167-73991-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:41.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[29496]: osdmap e295: 8 total, 8 up, 8 in 2024-06-18T13:08:41.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi167-73991-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:41.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi167-73991-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:41.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:41.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi167-73991-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:41.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi167-73991-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:41.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:41.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi167-73991-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:41.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[37701]: osdmap e295: 8 total, 8 up, 8 in 2024-06-18T13:08:41.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi167-73991-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:41.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi167-73991-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:41.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:43.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:42 smithi179 ceph-mon[33312]: pgmap v395: 292 pgs: 64 unknown, 228 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-06-18T13:08:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:43.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:42 smithi167 ceph-mon[29496]: pgmap v395: 292 pgs: 64 unknown, 228 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-06-18T13:08:43.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:43.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:42 smithi167 ceph-mon[37701]: pgmap v395: 292 pgs: 64 unknown, 228 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-06-18T13:08:43.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:08:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:43 smithi179 ceph-mon[33312]: osdmap e296: 8 total, 8 up, 8 in 2024-06-18T13:08:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:08:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi167-73991-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi167-73991-56"}]': finished 2024-06-18T13:08:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:08:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:43 smithi179 ceph-mon[33312]: osdmap e297: 8 total, 8 up, 8 in 2024-06-18T13:08:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:08:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:08:44.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:08:44.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[29496]: osdmap e296: 8 total, 8 up, 8 in 2024-06-18T13:08:44.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:08:44.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi167-73991-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi167-73991-56"}]': finished 2024-06-18T13:08:44.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:08:44.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[29496]: osdmap e297: 8 total, 8 up, 8 in 2024-06-18T13:08:44.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:08:44.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:08:44.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:08:44.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[37701]: osdmap e296: 8 total, 8 up, 8 in 2024-06-18T13:08:44.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:08:44.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi167-73991-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi167-73991-56"}]': finished 2024-06-18T13:08:44.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:08:44.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[37701]: osdmap e297: 8 total, 8 up, 8 in 2024-06-18T13:08:44.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:08:44.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:08:45.031 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:44 smithi167 ceph-mon[37701]: pgmap v398: 268 pgs: 8 creating+peering, 260 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:45.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:44 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:08:45.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:44 smithi167 ceph-mon[37701]: osdmap e298: 8 total, 8 up, 8 in 2024-06-18T13:08:45.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:08:45.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:44 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:08:45.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:44 smithi167 ceph-mon[29496]: pgmap v398: 268 pgs: 8 creating+peering, 260 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:45.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:44 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:08:45.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:44 smithi167 ceph-mon[29496]: osdmap e298: 8 total, 8 up, 8 in 2024-06-18T13:08:45.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:08:45.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:44 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:08:45.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:44 smithi179 ceph-mon[33312]: pgmap v398: 268 pgs: 8 creating+peering, 260 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:44 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:08:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:44 smithi179 ceph-mon[33312]: osdmap e298: 8 total, 8 up, 8 in 2024-06-18T13:08:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:08:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:44 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:08:45.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:08:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:08:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:08:46.891 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-18T13:08:46.891 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:46 smithi179 ceph-mon[33312]: osdmap e299: 8 total, 8 up, 8 in 2024-06-18T13:08:46.891 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:46.891 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:46.891 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:46 smithi179 ceph-mon[33312]: pgmap v401: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-18T13:08:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:46 smithi167 ceph-mon[37701]: osdmap e299: 8 total, 8 up, 8 in 2024-06-18T13:08:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:46 smithi167 ceph-mon[37701]: pgmap v401: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-18T13:08:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:46 smithi167 ceph-mon[29496]: osdmap e299: 8 total, 8 up, 8 in 2024-06-18T13:08:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:46 smithi167 ceph-mon[29496]: pgmap v401: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]': finished 2024-06-18T13:08:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:47 smithi179 ceph-mon[33312]: osdmap e300: 8 total, 8 up, 8 in 2024-06-18T13:08:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41"}]: dispatch 2024-06-18T13:08:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41"}]: dispatch 2024-06-18T13:08:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]': finished 2024-06-18T13:08:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[37701]: osdmap e300: 8 total, 8 up, 8 in 2024-06-18T13:08:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41"}]: dispatch 2024-06-18T13:08:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41"}]: dispatch 2024-06-18T13:08:47.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi167-73991-56"}]': finished 2024-06-18T13:08:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[29496]: osdmap e300: 8 total, 8 up, 8 in 2024-06-18T13:08:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2451460034' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]: dispatch 2024-06-18T13:08:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41"}]: dispatch 2024-06-18T13:08:47.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41"}]: dispatch 2024-06-18T13:08:49.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:08:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:08:49.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]': finished 2024-06-18T13:08:49.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41"}]': finished 2024-06-18T13:08:49.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:48 smithi179 ceph-mon[33312]: osdmap e301: 8 total, 8 up, 8 in 2024-06-18T13:08:49.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:49.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:49.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi167-73991-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:49.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:48 smithi179 ceph-mon[33312]: pgmap v404: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:49.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi167-73991-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:49.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:48 smithi179 ceph-mon[33312]: osdmap e302: 8 total, 8 up, 8 in 2024-06-18T13:08:49.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]': finished 2024-06-18T13:08:49.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41"}]': finished 2024-06-18T13:08:49.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[37701]: osdmap e301: 8 total, 8 up, 8 in 2024-06-18T13:08:49.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:49.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:49.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi167-73991-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:49.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[37701]: pgmap v404: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:49.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi167-73991-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:49.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[37701]: osdmap e302: 8 total, 8 up, 8 in 2024-06-18T13:08:49.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi167-73991-56"}]': finished 2024-06-18T13:08:49.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-41"}]': finished 2024-06-18T13:08:49.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[29496]: osdmap e301: 8 total, 8 up, 8 in 2024-06-18T13:08:49.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:49.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:49.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi167-73991-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:49.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[29496]: pgmap v404: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:49.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi167-73991-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:49.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:48 smithi167 ceph-mon[29496]: osdmap e302: 8 total, 8 up, 8 in 2024-06-18T13:08:49.741 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitS read_operations: Running main() from gmock_main.cc 2024-06-18T13:08:49.741 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-06-18T13:08:49.741 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [----------] Global test environment set-up. 2024-06-18T13:08:49.741 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-06-18T13:08:49.741 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-06-18T13:08:49.741 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (195976 ms) 2024-06-18T13:08:49.741 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-06-18T13:08:49.741 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (3014 ms) 2024-06-18T13:08:49.741 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-06-18T13:08:49.741 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (3002 ms) 2024-06-18T13:08:49.741 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-06-18T13:08:49.742 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (3007 ms) 2024-06-18T13:08:49.742 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-06-18T13:08:49.742 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (3011 ms) 2024-06-18T13:08:49.742 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-06-18T13:08:49.742 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (3014 ms) 2024-06-18T13:08:49.742 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-06-18T13:08:49.742 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (3152 ms) 2024-06-18T13:08:49.742 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-06-18T13:08:49.742 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (2763 ms) 2024-06-18T13:08:49.742 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-06-18T13:08:49.742 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (3028 ms) 2024-06-18T13:08:49.742 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-06-18T13:08:49.742 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (3003 ms) 2024-06-18T13:08:49.743 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-06-18T13:08:49.743 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (3016 ms) 2024-06-18T13:08:49.743 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-06-18T13:08:49.743 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (3009 ms) 2024-06-18T13:08:49.743 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-06-18T13:08:49.743 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (3006 ms) 2024-06-18T13:08:49.743 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-06-18T13:08:49.743 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (3243 ms) 2024-06-18T13:08:49.743 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (235244 ms total) 2024-06-18T13:08:49.743 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: 2024-06-18T13:08:49.743 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [----------] Global test environment tear-down 2024-06-18T13:08:49.743 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (235244 ms total) 2024-06-18T13:08:49.743 INFO:tasks.workunit.client.0.smithi167.stdout: read_operations: [ PASSED ] 14 tests. 2024-06-18T13:08:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi167-73991-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:49 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:49 smithi179 ceph-mon[33312]: osdmap e303: 8 total, 8 up, 8 in 2024-06-18T13:08:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:50.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi167-73991-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:50.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:49 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:50.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:49 smithi167 ceph-mon[37701]: osdmap e303: 8 total, 8 up, 8 in 2024-06-18T13:08:50.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:50.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi167-73991-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:49 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:49 smithi167 ceph-mon[29496]: osdmap e303: 8 total, 8 up, 8 in 2024-06-18T13:08:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:51.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:50 smithi179 ceph-mon[33312]: pgmap v407: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi167-73991-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi167-73991-57"}]': finished 2024-06-18T13:08:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:50 smithi179 ceph-mon[33312]: osdmap e304: 8 total, 8 up, 8 in 2024-06-18T13:08:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi167-74266-6","var": "pg_num","format": "json"}]: dispatch 2024-06-18T13:08:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:51.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:50 smithi167 ceph-mon[37701]: pgmap v407: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:51.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi167-73991-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi167-73991-57"}]': finished 2024-06-18T13:08:51.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:51.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:50 smithi167 ceph-mon[37701]: osdmap e304: 8 total, 8 up, 8 in 2024-06-18T13:08:51.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi167-74266-6","var": "pg_num","format": "json"}]: dispatch 2024-06-18T13:08:51.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:51.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:50 smithi167 ceph-mon[29496]: pgmap v407: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi167-73991-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi167-73991-57"}]': finished 2024-06-18T13:08:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:51.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:50 smithi167 ceph-mon[29496]: osdmap e304: 8 total, 8 up, 8 in 2024-06-18T13:08:51.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi167-74266-6","var": "pg_num","format": "json"}]: dispatch 2024-06-18T13:08:51.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:51.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:52.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:52 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:52 smithi179 ceph-mon[33312]: osdmap e305: 8 total, 8 up, 8 in 2024-06-18T13:08:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:52 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_count","val": "8"}]: dispatch 2024-06-18T13:08:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:52 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_count","val": "8"}]: dispatch 2024-06-18T13:08:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:52 smithi179 ceph-mon[33312]: pgmap v410: 268 pgs: 40 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:52 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:52 smithi167 ceph-mon[29496]: osdmap e305: 8 total, 8 up, 8 in 2024-06-18T13:08:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:52 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_count","val": "8"}]: dispatch 2024-06-18T13:08:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:52 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_count","val": "8"}]: dispatch 2024-06-18T13:08:52.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:52 smithi167 ceph-mon[29496]: pgmap v410: 268 pgs: 40 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:52.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:52 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:08:52.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:52 smithi167 ceph-mon[37701]: osdmap e305: 8 total, 8 up, 8 in 2024-06-18T13:08:52.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:52 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_count","val": "8"}]: dispatch 2024-06-18T13:08:52.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:52 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_count","val": "8"}]: dispatch 2024-06-18T13:08:52.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:52 smithi167 ceph-mon[37701]: pgmap v410: 268 pgs: 40 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_count","val": "8"}]': finished 2024-06-18T13:08:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:53 smithi179 ceph-mon[33312]: osdmap e306: 8 total, 8 up, 8 in 2024-06-18T13:08:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:08:53.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:08:53.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_count","val": "8"}]': finished 2024-06-18T13:08:53.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:53 smithi167 ceph-mon[29496]: osdmap e306: 8 total, 8 up, 8 in 2024-06-18T13:08:53.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:53.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:08:53.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:08:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_count","val": "8"}]': finished 2024-06-18T13:08:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:53 smithi167 ceph-mon[37701]: osdmap e306: 8 total, 8 up, 8 in 2024-06-18T13:08:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:53.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:08:53.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:08:54.752 INFO:tasks.workunit.client.0.smithi167.stdout:etRead 2024-06-18T13:08:54.752 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: hmm, no HitSet yet 2024-06-18T13:08:54.753 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: ok, hit_set contains 271:602f83fe:::foo:head 2024-06-18T13:08:54.753 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9023 ms) 2024-06-18T13:08:54.753 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-06-18T13:08:54.753 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg_num = 32 2024-06-18T13:08:54.753 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 0 ls 1718716135,0 2024-06-18T13:08:54.753 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 1 ls 1718716135,0 2024-06-18T13:08:54.753 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 2 ls 1718716135,0 2024-06-18T13:08:54.753 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 3 ls 1718716135,0 2024-06-18T13:08:54.753 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 4 ls 1718716135,0 2024-06-18T13:08:54.753 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 5 ls 1718716135,0 2024-06-18T13:08:54.753 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 6 ls 1718716135,0 2024-06-18T13:08:54.753 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 7 ls 1718716135,0 2024-06-18T13:08:54.753 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 8 ls 1718716135,0 2024-06-18T13:08:54.754 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 9 ls 1718716135,0 2024-06-18T13:08:54.754 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 10 ls 1718716135,0 2024-06-18T13:08:54.754 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 11 ls 1718716135,0 2024-06-18T13:08:54.754 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 12 ls 1718716135,0 2024-06-18T13:08:54.754 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 13 ls 1718716135,0 2024-06-18T13:08:54.754 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 14 ls 1718716135,0 2024-06-18T13:08:54.754 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 15 ls 1718716135,0 2024-06-18T13:08:54.754 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 16 ls 1718716135,0 2024-06-18T13:08:54.754 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 17 ls 1718716135,0 2024-06-18T13:08:54.754 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 18 ls 1718716135,0 2024-06-18T13:08:54.754 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 19 ls 1718716135,0 2024-06-18T13:08:54.754 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 20 ls 1718716135,0 2024-06-18T13:08:54.755 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 21 ls 1718716135,0 2024-06-18T13:08:54.755 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 22 ls 1718716135,0 2024-06-18T13:08:54.755 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 23 ls 1718716135,0 2024-06-18T13:08:54.755 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 24 ls 1718716135,0 2024-06-18T13:08:54.755 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 25 ls 1718716135,0 2024-06-18T13:08:54.755 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 26 ls 1718716135,0 2024-06-18T13:08:54.755 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 27 ls 1718716135,0 2024-06-18T13:08:54.755 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 28 ls 1718716135,0 2024-06-18T13:08:54.755 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 29 ls 1718716135,0 2024-06-18T13:08:54.755 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 30 ls 1718716135,0 2024-06-18T13:08:54.755 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg 31 ls 1718716135,0 2024-06-18T13:08:54.755 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: pg_num = 32 2024-06-18T13:08:54.755 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:6cac518f:::0:head 2024-06-18T13:08:54.755 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:02547ec2:::1:head 2024-06-18T13:08:54.756 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:f905c69b:::2:head 2024-06-18T13:08:54.756 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:cfc208b3:::3:head 2024-06-18T13:08:54.756 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:d83876eb:::4:head 2024-06-18T13:08:54.756 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:b29083e3:::5:head 2024-06-18T13:08:54.756 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:c4fdafeb:::6:head 2024-06-18T13:08:54.756 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:5c6b0b28:::7:head 2024-06-18T13:08:54.756 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:bd63b0f1:::8:head 2024-06-18T13:08:54.756 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:e960b815:::9:head 2024-06-18T13:08:54.756 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:52ea6a34:::10:head 2024-06-18T13:08:54.756 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:89d3ae78:::11:head 2024-06-18T13:08:54.756 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:de5d7c5f:::12:head 2024-06-18T13:08:54.757 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:566253c9:::13:head 2024-06-18T13:08:54.757 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:62a1935d:::14:head 2024-06-18T13:08:54.757 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:863748b0:::15:head 2024-06-18T13:08:54.757 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:3958e169:::16:head 2024-06-18T13:08:54.757 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:4d4dabf9:::17:head 2024-06-18T13:08:54.757 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:8391935d:::18:head 2024-06-18T13:08:54.757 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:28883081:::19:head 2024-06-18T13:08:54.757 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:69259c59:::20:head 2024-06-18T13:08:54.757 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:4bdb80b7:::21:head 2024-06-18T13:08:54.757 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:a11c5d71:::22:head 2024-06-18T13:08:54.757 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:271af37b:::23:head 2024-06-18T13:08:54.757 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:95b121be:::24:head 2024-06-18T13:08:54.757 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:58d1031b:::25:head 2024-06-18T13:08:54.757 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:0a050783:::26:head 2024-06-18T13:08:54.758 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:c709704c:::27:head 2024-06-18T13:08:54.758 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:cbe56eaf:::28:head 2024-06-18T13:08:54.758 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:86b4b162:::29:head 2024-06-18T13:08:54.758 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:70d89383:::30:head 2024-06-18T13:08:54.758 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:dd450c7c:::31:head 2024-06-18T13:08:54.758 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:6d5729b1:::32:head 2024-06-18T13:08:54.758 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:c388f3fb:::33:head 2024-06-18T13:08:54.758 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:56cfea31:::34:head 2024-06-18T13:08:54.758 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:9dbc1bf7:::35:head 2024-06-18T13:08:54.758 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:40b74ccd:::36:head 2024-06-18T13:08:54.758 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:4d5aaf42:::37:head 2024-06-18T13:08:54.758 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:920f362c:::38:head 2024-06-18T13:08:54.759 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:6cc53222:::39:head 2024-06-18T13:08:54.759 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:9cad833f:::40:head 2024-06-18T13:08:54.759 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:1ea84d41:::41:head 2024-06-18T13:08:54.759 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:c4480ef6:::42:head 2024-06-18T13:08:54.759 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:a694361e:::43:head 2024-06-18T13:08:54.759 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:d1bd33e9:::44:head 2024-06-18T13:08:54.759 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:ddc2cd5d:::45:head 2024-06-18T13:08:54.759 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:2b782207:::46:head 2024-06-18T13:08:54.759 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:7b187fca:::47:head 2024-06-18T13:08:54.759 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:90ecdf6f:::48:head 2024-06-18T13:08:54.759 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:a5ed95fe:::49:head 2024-06-18T13:08:54.759 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:ea0eaa55:::50:head 2024-06-18T13:08:54.759 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:f33ef17b:::51:head 2024-06-18T13:08:54.760 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:a0d1b2f6:::52:head 2024-06-18T13:08:54.760 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:60c5229e:::53:head 2024-06-18T13:08:54.760 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:edcbc575:::54:head 2024-06-18T13:08:54.760 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:102cf253:::55:head 2024-06-18T13:08:54.760 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:efb7fb0b:::56:head 2024-06-18T13:08:54.760 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:50d0a326:::57:head 2024-06-18T13:08:54.760 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:d4dc5daf:::58:head 2024-06-18T13:08:54.760 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:3a130462:::59:head 2024-06-18T13:08:54.760 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:ec87ed71:::60:head 2024-06-18T13:08:54.760 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:d5bc9454:::61:head 2024-06-18T13:08:54.760 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:3ddfe313:::62:head 2024-06-18T13:08:54.760 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:7c2816b9:::63:head 2024-06-18T13:08:54.760 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:47e00e4d:::64:head 2024-06-18T13:08:54.760 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:c6410c18:::65:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:b48ed237:::66:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:cd63ad31:::67:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:b179e92b:::68:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:0d9f741a:::69:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:6d3352ae:::70:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:c6d5c19e:::71:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:bc4729c3:::72:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:77e930b9:::73:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:0abeecfd:::74:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:b7c37e15:::75:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:b6378398:::76:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:02bd68de:::77:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:cc795d2d:::78:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:630d4fea:::79:head 2024-06-18T13:08:54.761 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:e0d29ef5:::80:head 2024-06-18T13:08:54.762 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:fd6f13d2:::81:head 2024-06-18T13:08:54.762 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:606461d5:::82:head 2024-06-18T13:08:54.762 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:eadbdc43:::83:head 2024-06-18T13:08:54.762 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:8761d0bb:::84:head 2024-06-18T13:08:54.762 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:9ef0186f:::85:head 2024-06-18T13:08:54.762 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:e0d41294:::86:head 2024-06-18T13:08:54.762 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:961de695:::87:head 2024-06-18T13:08:54.762 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:1423148f:::88:head 2024-06-18T13:08:54.762 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:633a8fa2:::89:head 2024-06-18T13:08:54.762 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:a8653809:::90:head 2024-06-18T13:08:54.762 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:3dac8b33:::91:head 2024-06-18T13:08:54.762 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:35aad435:::92:head 2024-06-18T13:08:54.762 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:f6dcc343:::93:head 2024-06-18T13:08:54.762 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:dbbdad87:::94:head 2024-06-18T13:08:54.763 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:1cb48ce0:::95:head 2024-06-18T13:08:54.763 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:03cd461c:::96:head 2024-06-18T13:08:54.763 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:17a4ea99:::97:head 2024-06-18T13:08:54.763 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:9993c9a7:::98:head 2024-06-18T13:08:54.763 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:6394211c:::99:head 2024-06-18T13:08:54.763 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:94c7ae57:::100:head 2024-06-18T13:08:54.763 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:6fdee5bb:::101:head 2024-06-18T13:08:54.763 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:9a477fd1:::102:head 2024-06-18T13:08:54.763 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:eb850916:::103:head 2024-06-18T13:08:54.763 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:affc56b9:::104:head 2024-06-18T13:08:54.763 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:b42dc814:::105:head 2024-06-18T13:08:54.763 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:f319f8f0:::106:head 2024-06-18T13:08:54.763 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:9a40b9de:::107:head 2024-06-18T13:08:54.763 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:8b524f28:::108:head 2024-06-18T13:08:54.764 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:e3de589f:::109:head 2024-06-18T13:08:54.764 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:90f90a5b:::110:head 2024-06-18T13:08:54.764 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:a7b4f1d7:::111:head 2024-06-18T13:08:54.764 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:af51766e:::112:head 2024-06-18T13:08:54.764 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:b6f90bd1:::113:head 2024-06-18T13:08:54.764 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:e0261208:::114:head 2024-06-18T13:08:54.764 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:c9569ef7:::115:head 2024-06-18T13:08:54.764 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:61bebe50:::116:head 2024-06-18T13:08:54.764 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:fe93412b:::117:head 2024-06-18T13:08:54.764 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:d3d38bee:::118:head 2024-06-18T13:08:54.764 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:3100ba0c:::119:head 2024-06-18T13:08:54.764 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:d0560ada:::120:head 2024-06-18T13:08:54.764 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:f0ea8b35:::121:head 2024-06-18T13:08:54.764 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:766f231a:::122:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:a07a2582:::123:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:bd7c6b3a:::124:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:fb2ddaff:::125:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:4408e1fe:::126:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:ee1df7a7:::127:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:c3002909:::128:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:4f48ffa9:::129:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:edf38733:::130:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:c08425c0:::131:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:5f902d98:::132:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:41ea2c93:::133:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:813cee13:::134:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:0131818d:::135:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:26ba5a85:::136:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:381b8a5a:::137:head 2024-06-18T13:08:54.765 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:28797e47:::138:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:bfca7f22:::139:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:36807075:::140:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:80b03975:::141:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:5c15709b:::142:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:f39ea15e:::143:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:ea992956:::144:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:48887b1c:::145:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:9f24a9dd:::146:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:987f100b:::147:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:d2dd3581:::148:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:7fed1808:::149:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:c80b70e9:::150:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:85ed90f9:::151:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:36428b24:::152:head 2024-06-18T13:08:54.766 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:d044c34a:::153:head 2024-06-18T13:08:54.767 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:7c18bf58:::154:head 2024-06-18T13:08:54.767 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:d1c21232:::155:head 2024-06-18T13:08:54.767 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:a7a3c575:::156:head 2024-06-18T13:08:54.767 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:87da0633:::157:head 2024-06-18T13:08:54.767 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:d5ac3822:::158:head 2024-06-18T13:08:54.767 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:3f20522d:::159:head 2024-06-18T13:08:54.767 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:6ca26563:::160:head 2024-06-18T13:08:54.767 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:532ce135:::161:head 2024-06-18T13:08:54.767 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:c78863e6:::162:head 2024-06-18T13:08:54.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi167-73991-57"}]': finished 2024-06-18T13:08:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:08:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:54 smithi179 ceph-mon[33312]: osdmap e307: 8 total, 8 up, 8 in 2024-06-18T13:08:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-06-18T13:08:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-06-18T13:08:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:54 smithi179 ceph-mon[33312]: pgmap v413: 260 pgs: 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:54 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:08:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi167-73991-57"}]': finished 2024-06-18T13:08:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-06-18T13:08:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:54 smithi179 ceph-mon[33312]: osdmap e308: 8 total, 8 up, 8 in 2024-06-18T13:08:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi167-73991-57"}]': finished 2024-06-18T13:08:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:08:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[37701]: osdmap e307: 8 total, 8 up, 8 in 2024-06-18T13:08:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-06-18T13:08:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-06-18T13:08:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[37701]: pgmap v413: 260 pgs: 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:08:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi167-73991-57"}]': finished 2024-06-18T13:08:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-06-18T13:08:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[37701]: osdmap e308: 8 total, 8 up, 8 in 2024-06-18T13:08:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi167-73991-57"}]': finished 2024-06-18T13:08:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:08:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[29496]: osdmap e307: 8 total, 8 up, 8 in 2024-06-18T13:08:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-06-18T13:08:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi167-73991-57"}]: dispatch 2024-06-18T13:08:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-06-18T13:08:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[29496]: pgmap v413: 260 pgs: 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:08:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2459697850' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi167-73991-57"}]': finished 2024-06-18T13:08:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-06-18T13:08:54.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:54 smithi167 ceph-mon[29496]: osdmap e308: 8 total, 8 up, 8 in 2024-06-18T13:08:55.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:08:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:08:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:55 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:08:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:08:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi167-73991-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi167-74266-43","var": "pg_num","format": "json"}]: dispatch 2024-06-18T13:08:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43"}]: dispatch 2024-06-18T13:08:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43"}]: dispatch 2024-06-18T13:08:55.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:08:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:08:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi167-73991-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi167-74266-43","var": "pg_num","format": "json"}]: dispatch 2024-06-18T13:08:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43"}]: dispatch 2024-06-18T13:08:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43"}]: dispatch 2024-06-18T13:08:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:08:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:08:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:08:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi167-73991-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:08:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi167-74266-43","var": "pg_num","format": "json"}]: dispatch 2024-06-18T13:08:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:08:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43"}]: dispatch 2024-06-18T13:08:55.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43"}]: dispatch 2024-06-18T13:08:56.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi167-73991-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:56.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43"}]': finished 2024-06-18T13:08:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:56 smithi179 ceph-mon[33312]: osdmap e309: 8 total, 8 up, 8 in 2024-06-18T13:08:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi167-73991-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:08:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:56 smithi179 ceph-mon[33312]: pgmap v416: 260 pgs: 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:56.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi167-73991-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43"}]': finished 2024-06-18T13:08:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:56 smithi167 ceph-mon[29496]: osdmap e309: 8 total, 8 up, 8 in 2024-06-18T13:08:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi167-73991-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:08:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:56 smithi167 ceph-mon[29496]: pgmap v416: 260 pgs: 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi167-73991-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:08:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-43"}]': finished 2024-06-18T13:08:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:56 smithi167 ceph-mon[37701]: osdmap e309: 8 total, 8 up, 8 in 2024-06-18T13:08:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi167-73991-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:08:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:56 smithi167 ceph-mon[37701]: pgmap v416: 260 pgs: 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:08:57.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:57 smithi179 ceph-mon[33312]: osdmap e310: 8 total, 8 up, 8 in 2024-06-18T13:08:57.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:57 smithi167 ceph-mon[37701]: osdmap e310: 8 total, 8 up, 8 in 2024-06-18T13:08:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:57 smithi167 ceph-mon[29496]: osdmap e310: 8 total, 8 up, 8 in 2024-06-18T13:08:58.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi167-73991-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi167-73991-58"}]': finished 2024-06-18T13:08:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:58 smithi179 ceph-mon[33312]: osdmap e311: 8 total, 8 up, 8 in 2024-06-18T13:08:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:58 smithi179 ceph-mon[33312]: pgmap v419: 268 pgs: 40 unknown, 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:58.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:08:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:08:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi167-73991-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi167-73991-58"}]': finished 2024-06-18T13:08:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:58 smithi167 ceph-mon[37701]: osdmap e311: 8 total, 8 up, 8 in 2024-06-18T13:08:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:58 smithi167 ceph-mon[37701]: pgmap v419: 268 pgs: 40 unknown, 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi167-73991-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi167-73991-58"}]': finished 2024-06-18T13:08:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:58 smithi167 ceph-mon[29496]: osdmap e311: 8 total, 8 up, 8 in 2024-06-18T13:08:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:08:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:58 smithi167 ceph-mon[29496]: pgmap v419: 268 pgs: 40 unknown, 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:08:59.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:59.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:59 smithi179 ceph-mon[33312]: osdmap e312: 8 total, 8 up, 8 in 2024-06-18T13:08:59.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:59 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:59.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:59.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:08:59 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:59.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:59 smithi167 ceph-mon[37701]: osdmap e312: 8 total, 8 up, 8 in 2024-06-18T13:08:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:59 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:08:59 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:08:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:08:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:59 smithi167 ceph-mon[29496]: osdmap e312: 8 total, 8 up, 8 in 2024-06-18T13:08:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:59 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:08:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:08:59 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:09:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:00 smithi179 ceph-mon[33312]: osdmap e313: 8 total, 8 up, 8 in 2024-06-18T13:09:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_count","val": "3"}]: dispatch 2024-06-18T13:09:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_count","val": "3"}]: dispatch 2024-06-18T13:09:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:09:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:00 smithi179 ceph-mon[33312]: pgmap v422: 260 pgs: 1 creating+activating, 31 creating+peering, 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_count","val": "3"}]': finished 2024-06-18T13:09:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:09:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:00 smithi167 ceph-mon[37701]: osdmap e313: 8 total, 8 up, 8 in 2024-06-18T13:09:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_count","val": "3"}]: dispatch 2024-06-18T13:09:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_count","val": "3"}]: dispatch 2024-06-18T13:09:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:09:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:00 smithi167 ceph-mon[37701]: pgmap v422: 260 pgs: 1 creating+activating, 31 creating+peering, 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_count","val": "3"}]': finished 2024-06-18T13:09:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:09:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:00 smithi167 ceph-mon[29496]: osdmap e313: 8 total, 8 up, 8 in 2024-06-18T13:09:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_count","val": "3"}]: dispatch 2024-06-18T13:09:00.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_count","val": "3"}]: dispatch 2024-06-18T13:09:00.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:09:00.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:00 smithi167 ceph-mon[29496]: pgmap v422: 260 pgs: 1 creating+activating, 31 creating+peering, 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:00.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_count","val": "3"}]': finished 2024-06-18T13:09:01.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi167-73991-58"}]': finished 2024-06-18T13:09:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:01 smithi179 ceph-mon[33312]: osdmap e314: 8 total, 8 up, 8 in 2024-06-18T13:09:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:01 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:09:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_period","val": "3"}]: dispatch 2024-06-18T13:09:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_period","val": "3"}]: dispatch 2024-06-18T13:09:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi167-73991-58"}]': finished 2024-06-18T13:09:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_period","val": "3"}]': finished 2024-06-18T13:09:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:01 smithi179 ceph-mon[33312]: osdmap e315: 8 total, 8 up, 8 in 2024-06-18T13:09:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi167-73991-58"}]': finished 2024-06-18T13:09:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[29496]: osdmap e314: 8 total, 8 up, 8 in 2024-06-18T13:09:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:09:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_period","val": "3"}]: dispatch 2024-06-18T13:09:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_period","val": "3"}]: dispatch 2024-06-18T13:09:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi167-73991-58"}]': finished 2024-06-18T13:09:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_period","val": "3"}]': finished 2024-06-18T13:09:01.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[29496]: osdmap e315: 8 total, 8 up, 8 in 2024-06-18T13:09:01.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:01.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:01.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi167-73991-58"}]': finished 2024-06-18T13:09:01.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[37701]: osdmap e314: 8 total, 8 up, 8 in 2024-06-18T13:09:01.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:01.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi167-73991-58"}]: dispatch 2024-06-18T13:09:01.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_period","val": "3"}]: dispatch 2024-06-18T13:09:01.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_period","val": "3"}]: dispatch 2024-06-18T13:09:01.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1415967751' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi167-73991-58"}]': finished 2024-06-18T13:09:01.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_period","val": "3"}]': finished 2024-06-18T13:09:01.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[37701]: osdmap e315: 8 total, 8 up, 8 in 2024-06-18T13:09:01.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:01.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:02.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:02 smithi179 ceph-mon[33312]: pgmap v425: 260 pgs: 1 creating+activating, 31 creating+peering, 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:09:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:02 smithi179 ceph-mon[33312]: osdmap e316: 8 total, 8 up, 8 in 2024-06-18T13:09:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-18T13:09:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi167-73991-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-18T13:09:02.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi167-73991-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:02.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:02.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[37701]: pgmap v425: 260 pgs: 1 creating+activating, 31 creating+peering, 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:02.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:09:02.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:02.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[37701]: osdmap e316: 8 total, 8 up, 8 in 2024-06-18T13:09:02.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-18T13:09:02.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi167-73991-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-18T13:09:02.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi167-73991-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:02.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:02.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[29496]: pgmap v425: 260 pgs: 1 creating+activating, 31 creating+peering, 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:02.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:09:02.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:02.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[29496]: osdmap e316: 8 total, 8 up, 8 in 2024-06-18T13:09:02.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-18T13:09:02.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi167-73991-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:02.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-18T13:09:02.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi167-73991-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-06-18T13:09:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:04 smithi179 ceph-mon[33312]: osdmap e317: 8 total, 8 up, 8 in 2024-06-18T13:09:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:04 smithi179 ceph-mon[33312]: pgmap v428: 260 pgs: 260 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:04.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-06-18T13:09:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:04 smithi167 ceph-mon[29496]: osdmap e317: 8 total, 8 up, 8 in 2024-06-18T13:09:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:04 smithi167 ceph-mon[29496]: pgmap v428: 260 pgs: 260 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-06-18T13:09:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:04 smithi167 ceph-mon[37701]: osdmap e317: 8 total, 8 up, 8 in 2024-06-18T13:09:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:04 smithi167 ceph-mon[37701]: pgmap v428: 260 pgs: 260 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:05.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:09:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:09:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:09:05.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi167-73991-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]': finished 2024-06-18T13:09:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:05 smithi179 ceph-mon[33312]: osdmap e318: 8 total, 8 up, 8 in 2024-06-18T13:09:05.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi167-73991-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]': finished 2024-06-18T13:09:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:05 smithi167 ceph-mon[37701]: osdmap e318: 8 total, 8 up, 8 in 2024-06-18T13:09:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi167-73991-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]': finished 2024-06-18T13:09:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:05 smithi167 ceph-mon[29496]: osdmap e318: 8 total, 8 up, 8 in 2024-06-18T13:09:06.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:06 smithi179 ceph-mon[33312]: osdmap e319: 8 total, 8 up, 8 in 2024-06-18T13:09:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:06 smithi179 ceph-mon[33312]: pgmap v431: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:06.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:06 smithi167 ceph-mon[29496]: osdmap e319: 8 total, 8 up, 8 in 2024-06-18T13:09:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:06 smithi167 ceph-mon[29496]: pgmap v431: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:06 smithi167 ceph-mon[37701]: osdmap e319: 8 total, 8 up, 8 in 2024-06-18T13:09:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:06 smithi167 ceph-mon[37701]: pgmap v431: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:07.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:07 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:07.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:07 smithi179 ceph-mon[33312]: osdmap e320: 8 total, 8 up, 8 in 2024-06-18T13:09:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:07.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:07 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:07 smithi167 ceph-mon[29496]: osdmap e320: 8 total, 8 up, 8 in 2024-06-18T13:09:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:07 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:07 smithi167 ceph-mon[37701]: osdmap e320: 8 total, 8 up, 8 in 2024-06-18T13:09:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:08.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:7462ddf6::o.RoundTripAppendPP (3009 ms) 2024-06-18T13:09:08.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-06-18T13:09:08.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3001 ms) 2024-06-18T13:09:08.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-06-18T13:09:08.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3009 ms) 2024-06-18T13:09:08.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-06-18T13:09:08.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (2985 ms) 2024-06-18T13:09:08.599 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-06-18T13:09:08.599 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: setting pool EIO 2024-06-18T13:09:08.599 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: max_success 100, min_failed 101 2024-06-18T13:09:08.599 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4013 ms) 2024-06-18T13:09:08.599 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-06-18T13:09:08.599 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3007 ms) 2024-06-18T13:09:08.599 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (127386 ms total) 2024-06-18T13:09:08.599 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: 2024-06-18T13:09:08.599 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-06-18T13:09:08.599 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-06-18T13:09:08.599 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3250 ms) 2024-06-18T13:09:08.599 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-06-18T13:09:08.599 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (8799 ms) 2024-06-18T13:09:08.600 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-06-18T13:09:08.600 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14942 ms) 2024-06-18T13:09:08.600 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-06-18T13:09:08.600 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3005 ms) 2024-06-18T13:09:08.600 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29996 ms total) 2024-06-18T13:09:08.600 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: 2024-06-18T13:09:08.600 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-06-18T13:09:08.600 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-06-18T13:09:08.600 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3007 ms) 2024-06-18T13:09:08.600 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3007 ms total) 2024-06-18T13:09:08.600 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: 2024-06-18T13:09:08.600 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-06-18T13:09:08.600 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-06-18T13:09:08.600 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (14016 ms) 2024-06-18T13:09:08.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-06-18T13:09:08.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (7054 ms) 2024-06-18T13:09:08.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-06-18T13:09:08.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (6932 ms) 2024-06-18T13:09:08.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-06-18T13:09:08.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7042 ms) 2024-06-18T13:09:08.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-06-18T13:09:08.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3005 ms) 2024-06-18T13:09:08.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-06-18T13:09:08.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7017 ms) 2024-06-18T13:09:08.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-06-18T13:09:08.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (7023 ms) 2024-06-18T13:09:08.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-06-18T13:09:08.601 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (6927 ms) 2024-06-18T13:09:08.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-06-18T13:09:08.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7032 ms) 2024-06-18T13:09:08.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-06-18T13:09:08.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7036 ms) 2024-06-18T13:09:08.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-06-18T13:09:08.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7004 ms) 2024-06-18T13:09:08.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-06-18T13:09:08.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7053 ms) 2024-06-18T13:09:08.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-06-18T13:09:08.602 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7031 ms) 2024-06-18T13:09:08.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:09:08 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:09:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]': finished 2024-06-18T13:09:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:08 smithi179 ceph-mon[33312]: osdmap e321: 8 total, 8 up, 8 in 2024-06-18T13:09:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:08 smithi179 ceph-mon[33312]: pgmap v434: 260 pgs: 260 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]': finished 2024-06-18T13:09:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:08 smithi179 ceph-mon[33312]: osdmap e322: 8 total, 8 up, 8 in 2024-06-18T13:09:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]': finished 2024-06-18T13:09:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:08 smithi167 ceph-mon[37701]: osdmap e321: 8 total, 8 up, 8 in 2024-06-18T13:09:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:08 smithi167 ceph-mon[37701]: pgmap v434: 260 pgs: 260 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]': finished 2024-06-18T13:09:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:08 smithi167 ceph-mon[37701]: osdmap e322: 8 total, 8 up, 8 in 2024-06-18T13:09:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi167-73991-59"}]': finished 2024-06-18T13:09:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:08 smithi167 ceph-mon[29496]: osdmap e321: 8 total, 8 up, 8 in 2024-06-18T13:09:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1264090879' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]: dispatch 2024-06-18T13:09:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:08 smithi167 ceph-mon[29496]: pgmap v434: 260 pgs: 260 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi167-73991-59"}]': finished 2024-06-18T13:09:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:08 smithi167 ceph-mon[29496]: osdmap e322: 8 total, 8 up, 8 in 2024-06-18T13:09:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:09 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:09:09.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:09 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:09:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:09 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:09:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:10 smithi179 ceph-mon[33312]: osdmap e323: 8 total, 8 up, 8 in 2024-06-18T13:09:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/526502657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:10 smithi179 ceph-mon[33312]: pgmap v437: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:09:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/526502657' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:09:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:10 smithi179 ceph-mon[33312]: osdmap e324: 8 total, 8 up, 8 in 2024-06-18T13:09:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:10 smithi167 ceph-mon[37701]: osdmap e323: 8 total, 8 up, 8 in 2024-06-18T13:09:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/526502657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:10 smithi167 ceph-mon[37701]: pgmap v437: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:09:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/526502657' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:09:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:10 smithi167 ceph-mon[37701]: osdmap e324: 8 total, 8 up, 8 in 2024-06-18T13:09:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:10 smithi167 ceph-mon[29496]: osdmap e323: 8 total, 8 up, 8 in 2024-06-18T13:09:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/526502657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:10 smithi167 ceph-mon[29496]: pgmap v437: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:09:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/526502657' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi167-73991-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:09:10.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:10 smithi167 ceph-mon[29496]: osdmap e324: 8 total, 8 up, 8 in 2024-06-18T13:09:12.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:12 smithi179 ceph-mon[33312]: osdmap e325: 8 total, 8 up, 8 in 2024-06-18T13:09:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi167-73991-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi167-73991-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:12 smithi179 ceph-mon[33312]: pgmap v440: 260 pgs: 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:09:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[29496]: osdmap e325: 8 total, 8 up, 8 in 2024-06-18T13:09:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi167-73991-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi167-73991-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[29496]: pgmap v440: 260 pgs: 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:09:12.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[37701]: osdmap e325: 8 total, 8 up, 8 in 2024-06-18T13:09:12.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:12.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:12.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:12.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:12.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi167-73991-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:12.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi167-73991-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:12.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:12 smithi167 ceph-mon[37701]: pgmap v440: 260 pgs: 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:09:13.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:13 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi167-73991-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:13 smithi179 ceph-mon[33312]: osdmap e326: 8 total, 8 up, 8 in 2024-06-18T13:09:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi167-73991-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi167-73991-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:13.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:13 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi167-73991-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:13 smithi167 ceph-mon[37701]: osdmap e326: 8 total, 8 up, 8 in 2024-06-18T13:09:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi167-73991-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi167-73991-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:13 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi167-73991-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:13 smithi167 ceph-mon[29496]: osdmap e326: 8 total, 8 up, 8 in 2024-06-18T13:09:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi167-73991-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi167-73991-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:14.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:14 smithi167 ceph-mon[37701]: osdmap e327: 8 total, 8 up, 8 in 2024-06-18T13:09:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:14 smithi167 ceph-mon[37701]: pgmap v443: 260 pgs: 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-06-18T13:09:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:14 smithi167 ceph-mon[29496]: osdmap e327: 8 total, 8 up, 8 in 2024-06-18T13:09:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:14 smithi167 ceph-mon[29496]: pgmap v443: 260 pgs: 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-06-18T13:09:15.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:14 smithi179 ceph-mon[33312]: osdmap e327: 8 total, 8 up, 8 in 2024-06-18T13:09:15.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:14 smithi179 ceph-mon[33312]: pgmap v443: 260 pgs: 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-06-18T13:09:15.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:09:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:09:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:09:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi167-73991-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi167-73991-61"}]': finished 2024-06-18T13:09:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:15 smithi167 ceph-mon[29496]: osdmap e328: 8 total, 8 up, 8 in 2024-06-18T13:09:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45"}]: dispatch 2024-06-18T13:09:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45"}]: dispatch 2024-06-18T13:09:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi167-73991-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi167-73991-61"}]': finished 2024-06-18T13:09:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:15 smithi167 ceph-mon[37701]: osdmap e328: 8 total, 8 up, 8 in 2024-06-18T13:09:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:15.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:15.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45"}]: dispatch 2024-06-18T13:09:15.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45"}]: dispatch 2024-06-18T13:09:16.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi167-73991-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi167-73991-61"}]': finished 2024-06-18T13:09:16.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:15 smithi179 ceph-mon[33312]: osdmap e328: 8 total, 8 up, 8 in 2024-06-18T13:09:16.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:16.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:16.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45"}]: dispatch 2024-06-18T13:09:16.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45"}]: dispatch 2024-06-18T13:09:16.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45"}]': finished 2024-06-18T13:09:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:16 smithi167 ceph-mon[37701]: osdmap e329: 8 total, 8 up, 8 in 2024-06-18T13:09:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:16 smithi167 ceph-mon[37701]: pgmap v446: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-06-18T13:09:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45"}]': finished 2024-06-18T13:09:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:16 smithi167 ceph-mon[29496]: osdmap e329: 8 total, 8 up, 8 in 2024-06-18T13:09:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:16 smithi167 ceph-mon[29496]: pgmap v446: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-06-18T13:09:17.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-45"}]': finished 2024-06-18T13:09:17.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:16 smithi179 ceph-mon[33312]: osdmap e329: 8 total, 8 up, 8 in 2024-06-18T13:09:17.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:16 smithi179 ceph-mon[33312]: pgmap v446: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-06-18T13:09:17.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:17 smithi167 ceph-mon[37701]: osdmap e330: 8 total, 8 up, 8 in 2024-06-18T13:09:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:17 smithi167 ceph-mon[29496]: osdmap e330: 8 total, 8 up, 8 in 2024-06-18T13:09:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:18.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:17 smithi179 ceph-mon[33312]: osdmap e330: 8 total, 8 up, 8 in 2024-06-18T13:09:18.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:18.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]': finished 2024-06-18T13:09:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:18 smithi167 ceph-mon[37701]: osdmap e331: 8 total, 8 up, 8 in 2024-06-18T13:09:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:18 smithi167 ceph-mon[37701]: pgmap v449: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]': finished 2024-06-18T13:09:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:18 smithi167 ceph-mon[29496]: osdmap e331: 8 total, 8 up, 8 in 2024-06-18T13:09:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:18 smithi167 ceph-mon[29496]: pgmap v449: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:19.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:09:18 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:09:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi167-73991-61"}]': finished 2024-06-18T13:09:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:18 smithi179 ceph-mon[33312]: osdmap e331: 8 total, 8 up, 8 in 2024-06-18T13:09:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2365527001' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]: dispatch 2024-06-18T13:09:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:18 smithi179 ceph-mon[33312]: pgmap v449: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:19.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]': finished 2024-06-18T13:09:19.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:09:19.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[37701]: osdmap e332: 8 total, 8 up, 8 in 2024-06-18T13:09:19.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:19.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:19.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:19.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:19.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:19.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:19.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]': finished 2024-06-18T13:09:19.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:09:19.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[29496]: osdmap e332: 8 total, 8 up, 8 in 2024-06-18T13:09:19.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:19.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:19.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:19.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:19.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:19.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:19 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi167-73991-61"}]': finished 2024-06-18T13:09:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:09:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:19 smithi179 ceph-mon[33312]: osdmap e332: 8 total, 8 up, 8 in 2024-06-18T13:09:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:19 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:20.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:09:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:20 smithi167 ceph-mon[37701]: osdmap e333: 8 total, 8 up, 8 in 2024-06-18T13:09:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi167-73991-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:20 smithi167 ceph-mon[37701]: pgmap v452: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:09:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:20 smithi167 ceph-mon[29496]: osdmap e333: 8 total, 8 up, 8 in 2024-06-18T13:09:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi167-73991-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:20.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:20.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:20 smithi167 ceph-mon[29496]: pgmap v452: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:21.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:09:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:20 smithi179 ceph-mon[33312]: osdmap e333: 8 total, 8 up, 8 in 2024-06-18T13:09:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi167-73991-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:20 smithi179 ceph-mon[33312]: pgmap v452: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:21.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:21 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-47"}]': finished 2024-06-18T13:09:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-47", "mode": "writeback"}]: dispatch 2024-06-18T13:09:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:21 smithi167 ceph-mon[37701]: osdmap e334: 8 total, 8 up, 8 in 2024-06-18T13:09:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-47", "mode": "writeback"}]: dispatch 2024-06-18T13:09:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:21 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-47"}]': finished 2024-06-18T13:09:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-47", "mode": "writeback"}]: dispatch 2024-06-18T13:09:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:21 smithi167 ceph-mon[29496]: osdmap e334: 8 total, 8 up, 8 in 2024-06-18T13:09:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-47", "mode": "writeback"}]: dispatch 2024-06-18T13:09:22.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:21 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:22.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-47"}]': finished 2024-06-18T13:09:22.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-47", "mode": "writeback"}]: dispatch 2024-06-18T13:09:22.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:21 smithi179 ceph-mon[33312]: osdmap e334: 8 total, 8 up, 8 in 2024-06-18T13:09:22.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-47", "mode": "writeback"}]: dispatch 2024-06-18T13:09:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:22 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:09:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi167-73991-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi167-73991-62"}]': finished 2024-06-18T13:09:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-47", "mode": "writeback"}]': finished 2024-06-18T13:09:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:22 smithi167 ceph-mon[37701]: osdmap e335: 8 total, 8 up, 8 in 2024-06-18T13:09:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:09:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:09:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:22 smithi167 ceph-mon[37701]: pgmap v455: 268 pgs: 8 unknown, 32 creating+peering, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:22 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:09:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi167-73991-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi167-73991-62"}]': finished 2024-06-18T13:09:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-47", "mode": "writeback"}]': finished 2024-06-18T13:09:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:22 smithi167 ceph-mon[29496]: osdmap e335: 8 total, 8 up, 8 in 2024-06-18T13:09:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:09:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:09:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:22 smithi167 ceph-mon[29496]: pgmap v455: 268 pgs: 8 unknown, 32 creating+peering, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:23.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:22 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:09:23.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi167-73991-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi167-73991-62"}]': finished 2024-06-18T13:09:23.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-47", "mode": "writeback"}]': finished 2024-06-18T13:09:23.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:22 smithi179 ceph-mon[33312]: osdmap e335: 8 total, 8 up, 8 in 2024-06-18T13:09:23.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:09:23.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:09:23.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:22 smithi179 ceph-mon[33312]: pgmap v455: 268 pgs: 8 unknown, 32 creating+peering, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:24.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:09:24.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:09:24.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:23 smithi179 ceph-mon[33312]: osdmap e336: 8 total, 8 up, 8 in 2024-06-18T13:09:24.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:09:24.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:09:24.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:09:24.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:23 smithi167 ceph-mon[37701]: osdmap e336: 8 total, 8 up, 8 in 2024-06-18T13:09:24.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:09:24.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:09:24.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:09:24.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:23 smithi167 ceph-mon[29496]: osdmap e336: 8 total, 8 up, 8 in 2024-06-18T13:09:24.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:09:25.031 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:09:25.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[37701]: osdmap e337: 8 total, 8 up, 8 in 2024-06-18T13:09:25.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:25.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:25.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:25.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[37701]: pgmap v458: 260 pgs: 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-18T13:09:25.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:09:25.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:09:25.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:09:25.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[29496]: osdmap e337: 8 total, 8 up, 8 in 2024-06-18T13:09:25.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:25.033 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:25.033 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:25.033 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[29496]: pgmap v458: 260 pgs: 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-18T13:09:25.033 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:09:25.033 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:24 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:09:25.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:09:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:24 smithi179 ceph-mon[33312]: osdmap e337: 8 total, 8 up, 8 in 2024-06-18T13:09:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:24 smithi179 ceph-mon[33312]: pgmap v458: 260 pgs: 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-18T13:09:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:24 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:09:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:24 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:09:25.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:09:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:09:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:25 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:09:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62"}]': finished 2024-06-18T13:09:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:09:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:25 smithi179 ceph-mon[33312]: osdmap e338: 8 total, 8 up, 8 in 2024-06-18T13:09:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:09:26.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:26.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:09:26.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:09:26.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62"}]': finished 2024-06-18T13:09:26.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:09:26.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-mon[37701]: osdmap e338: 8 total, 8 up, 8 in 2024-06-18T13:09:26.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:09:26.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:26.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:09:26.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:09:26.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi167-73991-62"}]': finished 2024-06-18T13:09:26.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:09:26.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-mon[29496]: osdmap e338: 8 total, 8 up, 8 in 2024-06-18T13:09:26.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:09:26.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi167-73991-62"}]: dispatch 2024-06-18T13:09:26.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:09:27.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi167-73991-62"}]': finished 2024-06-18T13:09:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-18T13:09:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:26 smithi179 ceph-mon[33312]: osdmap e339: 8 total, 8 up, 8 in 2024-06-18T13:09:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-18T13:09:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-18T13:09:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi167-73991-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:26 smithi179 ceph-mon[33312]: pgmap v461: 260 pgs: 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-18T13:09:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi167-73991-62"}]': finished 2024-06-18T13:09:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-18T13:09:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[37701]: osdmap e339: 8 total, 8 up, 8 in 2024-06-18T13:09:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-18T13:09:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-18T13:09:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi167-73991-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[37701]: pgmap v461: 260 pgs: 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-18T13:09:27.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3958614375' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi167-73991-62"}]': finished 2024-06-18T13:09:27.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-18T13:09:27.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[29496]: osdmap e339: 8 total, 8 up, 8 in 2024-06-18T13:09:27.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-18T13:09:27.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-18T13:09:27.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:27.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:27.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi167-73991-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:27.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:26 smithi167 ceph-mon[29496]: pgmap v461: 260 pgs: 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-18T13:09:28.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-06-18T13:09:28.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi167-73991-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:28.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:28 smithi179 ceph-mon[33312]: osdmap e340: 8 total, 8 up, 8 in 2024-06-18T13:09:28.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-18T13:09:28.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi167-73991-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:28.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-18T13:09:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-06-18T13:09:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi167-73991-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:28 smithi167 ceph-mon[29496]: osdmap e340: 8 total, 8 up, 8 in 2024-06-18T13:09:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-18T13:09:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi167-73991-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-18T13:09:28.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-06-18T13:09:28.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi167-73991-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:28.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:28 smithi167 ceph-mon[37701]: osdmap e340: 8 total, 8 up, 8 in 2024-06-18T13:09:28.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-18T13:09:28.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi167-73991-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:28.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-18T13:09:29.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:09:28 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:09:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-06-18T13:09:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:29 smithi179 ceph-mon[33312]: osdmap e341: 8 total, 8 up, 8 in 2024-06-18T13:09:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:29 smithi179 ceph-mon[33312]: pgmap v464: 260 pgs: 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi167-73991-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi167-73991-63"}]': finished 2024-06-18T13:09:29.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:09:29.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:29 smithi179 ceph-mon[33312]: osdmap e342: 8 total, 8 up, 8 in 2024-06-18T13:09:29.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:29.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-06-18T13:09:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[37701]: osdmap e341: 8 total, 8 up, 8 in 2024-06-18T13:09:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[37701]: pgmap v464: 260 pgs: 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi167-73991-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi167-73991-63"}]': finished 2024-06-18T13:09:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:09:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[37701]: osdmap e342: 8 total, 8 up, 8 in 2024-06-18T13:09:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:29.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:29.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-06-18T13:09:29.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[29496]: osdmap e341: 8 total, 8 up, 8 in 2024-06-18T13:09:29.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:29.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:29.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[29496]: pgmap v464: 260 pgs: 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:29.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi167-73991-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi167-73991-63"}]': finished 2024-06-18T13:09:29.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:09:29.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[29496]: osdmap e342: 8 total, 8 up, 8 in 2024-06-18T13:09:29.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:29.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:30.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:30.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]': finished 2024-06-18T13:09:30.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:30 smithi179 ceph-mon[33312]: osdmap e343: 8 total, 8 up, 8 in 2024-06-18T13:09:30.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:30.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:30.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:30.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:30.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:30.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]': finished 2024-06-18T13:09:30.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:30 smithi167 ceph-mon[37701]: osdmap e343: 8 total, 8 up, 8 in 2024-06-18T13:09:30.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:30.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:30.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:30.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:30.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:30.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]': finished 2024-06-18T13:09:30.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:30 smithi167 ceph-mon[29496]: osdmap e343: 8 total, 8 up, 8 in 2024-06-18T13:09:30.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:30.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:30.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:30.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-47"}]: dispatch 2024-06-18T13:09:31.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:31 smithi179 ceph-mon[33312]: pgmap v467: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:09:31.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:31 smithi179 ceph-mon[33312]: osdmap e344: 8 total, 8 up, 8 in 2024-06-18T13:09:31.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:31.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:31 smithi167 ceph-mon[37701]: pgmap v467: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:09:31.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:31 smithi167 ceph-mon[37701]: osdmap e344: 8 total, 8 up, 8 in 2024-06-18T13:09:31.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:31.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:31 smithi167 ceph-mon[29496]: pgmap v467: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:09:31.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:31 smithi167 ceph-mon[29496]: osdmap e344: 8 total, 8 up, 8 in 2024-06-18T13:09:31.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:32.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:32 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:32.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi167-73991-63"}]': finished 2024-06-18T13:09:32.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:32 smithi179 ceph-mon[33312]: osdmap e345: 8 total, 8 up, 8 in 2024-06-18T13:09:32.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:32.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:32.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:32.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:32 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:32.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi167-73991-63"}]': finished 2024-06-18T13:09:32.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:32 smithi167 ceph-mon[37701]: osdmap e345: 8 total, 8 up, 8 in 2024-06-18T13:09:32.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:32.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:32.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:32.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:32 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:32.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi167-73991-63"}]': finished 2024-06-18T13:09:32.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:32 smithi167 ceph-mon[29496]: osdmap e345: 8 total, 8 up, 8 in 2024-06-18T13:09:32.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi167-73991-63"}]: dispatch 2024-06-18T13:09:32.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:32.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:33.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:33 smithi179 ceph-mon[33312]: pgmap v470: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:09:33.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi167-73991-63"}]': finished 2024-06-18T13:09:33.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:33 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:09:33.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:33 smithi179 ceph-mon[33312]: osdmap e346: 8 total, 8 up, 8 in 2024-06-18T13:09:33.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:33.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:33 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:33.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:33.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:33.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi167-73991-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:33.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[37701]: pgmap v470: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:09:33.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi167-73991-63"}]': finished 2024-06-18T13:09:33.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:09:33.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[37701]: osdmap e346: 8 total, 8 up, 8 in 2024-06-18T13:09:33.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:33.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:33.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:33.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:33.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi167-73991-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:33.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[29496]: pgmap v470: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:09:33.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2689281168' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi167-73991-63"}]': finished 2024-06-18T13:09:33.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:09:33.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[29496]: osdmap e346: 8 total, 8 up, 8 in 2024-06-18T13:09:33.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:33.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:33.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:33.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:33.480 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi167-73991-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:35.031 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:09:35.031 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi167-73991-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:35.031 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:34 smithi167 ceph-mon[37701]: osdmap e347: 8 total, 8 up, 8 in 2024-06-18T13:09:35.031 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi167-73991-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:35.031 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:35.031 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:35.031 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:34 smithi167 ceph-mon[37701]: pgmap v473: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:09:35.031 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:09:35.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi167-73991-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:35.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:34 smithi167 ceph-mon[29496]: osdmap e347: 8 total, 8 up, 8 in 2024-06-18T13:09:35.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi167-73991-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:35.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:35.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:35.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:34 smithi167 ceph-mon[29496]: pgmap v473: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:09:35.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:09:35.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi167-73991-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:34 smithi179 ceph-mon[33312]: osdmap e347: 8 total, 8 up, 8 in 2024-06-18T13:09:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi167-73991-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:34 smithi179 ceph-mon[33312]: pgmap v473: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:09:35.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:09:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:09:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:09:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-49"}]': finished 2024-06-18T13:09:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-49", "mode": "readproxy"}]: dispatch 2024-06-18T13:09:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:35 smithi179 ceph-mon[33312]: osdmap e348: 8 total, 8 up, 8 in 2024-06-18T13:09:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-49", "mode": "readproxy"}]: dispatch 2024-06-18T13:09:36.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-49"}]': finished 2024-06-18T13:09:36.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-49", "mode": "readproxy"}]: dispatch 2024-06-18T13:09:36.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:35 smithi167 ceph-mon[37701]: osdmap e348: 8 total, 8 up, 8 in 2024-06-18T13:09:36.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-49", "mode": "readproxy"}]: dispatch 2024-06-18T13:09:36.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-49"}]': finished 2024-06-18T13:09:36.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-49", "mode": "readproxy"}]: dispatch 2024-06-18T13:09:36.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:35 smithi167 ceph-mon[29496]: osdmap e348: 8 total, 8 up, 8 in 2024-06-18T13:09:36.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-49", "mode": "readproxy"}]: dispatch 2024-06-18T13:09:37.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:36 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:09:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi167-73991-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi167-73991-64"}]': finished 2024-06-18T13:09:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-49", "mode": "readproxy"}]': finished 2024-06-18T13:09:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:36 smithi179 ceph-mon[33312]: osdmap e349: 8 total, 8 up, 8 in 2024-06-18T13:09:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:36 smithi179 ceph-mon[33312]: pgmap v476: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:09:37.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:36 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:09:37.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi167-73991-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi167-73991-64"}]': finished 2024-06-18T13:09:37.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-49", "mode": "readproxy"}]': finished 2024-06-18T13:09:37.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:36 smithi167 ceph-mon[37701]: osdmap e349: 8 total, 8 up, 8 in 2024-06-18T13:09:37.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:36 smithi167 ceph-mon[37701]: pgmap v476: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:09:37.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:36 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:09:37.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi167-73991-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi167-73991-64"}]': finished 2024-06-18T13:09:37.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-49", "mode": "readproxy"}]': finished 2024-06-18T13:09:37.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:36 smithi167 ceph-mon[29496]: osdmap e349: 8 total, 8 up, 8 in 2024-06-18T13:09:37.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:36 smithi167 ceph-mon[29496]: pgmap v476: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:09:38.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:37 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:38.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:37 smithi179 ceph-mon[33312]: osdmap e350: 8 total, 8 up, 8 in 2024-06-18T13:09:38.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:37 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:38.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:37 smithi167 ceph-mon[37701]: osdmap e350: 8 total, 8 up, 8 in 2024-06-18T13:09:38.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:37 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:38.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:37 smithi167 ceph-mon[29496]: osdmap e350: 8 total, 8 up, 8 in 2024-06-18T13:09:39.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:38 smithi179 ceph-mon[33312]: osdmap e351: 8 total, 8 up, 8 in 2024-06-18T13:09:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:38 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:38 smithi179 ceph-mon[33312]: pgmap v479: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:39.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:09:38 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:09:39.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:38 smithi167 ceph-mon[37701]: osdmap e351: 8 total, 8 up, 8 in 2024-06-18T13:09:39.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:38 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:39.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:38 smithi167 ceph-mon[37701]: pgmap v479: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:39.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:38 smithi167 ceph-mon[29496]: osdmap e351: 8 total, 8 up, 8 in 2024-06-18T13:09:39.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:38 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:39.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:38 smithi167 ceph-mon[29496]: pgmap v479: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:40.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:39 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:40.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi167-73991-64"}]': finished 2024-06-18T13:09:40.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:39 smithi179 ceph-mon[33312]: osdmap e352: 8 total, 8 up, 8 in 2024-06-18T13:09:40.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:40.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:09:40.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:09:40.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:39 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:40.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi167-73991-64"}]': finished 2024-06-18T13:09:40.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:39 smithi167 ceph-mon[37701]: osdmap e352: 8 total, 8 up, 8 in 2024-06-18T13:09:40.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:40.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:09:40.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:09:40.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:39 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:40.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi167-73991-64"}]': finished 2024-06-18T13:09:40.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:39 smithi167 ceph-mon[29496]: osdmap e352: 8 total, 8 up, 8 in 2024-06-18T13:09:40.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi167-73991-64"}]: dispatch 2024-06-18T13:09:40.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:09:40.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:09:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi167-73991-64"}]': finished 2024-06-18T13:09:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:40 smithi179 ceph-mon[33312]: osdmap e353: 8 total, 8 up, 8 in 2024-06-18T13:09:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi167-73991-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:09:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:09:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:09:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:40 smithi179 ceph-mon[33312]: pgmap v482: 260 pgs: 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:09:41.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi167-73991-64"}]': finished 2024-06-18T13:09:41.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[37701]: osdmap e353: 8 total, 8 up, 8 in 2024-06-18T13:09:41.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:41.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:41.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi167-73991-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:41.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:09:41.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:09:41.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:09:41.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[37701]: pgmap v482: 260 pgs: 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:09:41.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2764472531' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi167-73991-64"}]': finished 2024-06-18T13:09:41.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[29496]: osdmap e353: 8 total, 8 up, 8 in 2024-06-18T13:09:41.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:41.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:41.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi167-73991-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:41.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:09:41.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:09:41.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:09:41.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:40 smithi167 ceph-mon[29496]: pgmap v482: 260 pgs: 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:09:42.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi167-73991-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:42.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:42 smithi179 ceph-mon[33312]: osdmap e354: 8 total, 8 up, 8 in 2024-06-18T13:09:42.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi167-73991-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:42.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi167-73991-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:42.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:42 smithi167 ceph-mon[37701]: osdmap e354: 8 total, 8 up, 8 in 2024-06-18T13:09:42.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi167-73991-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:42.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi167-73991-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:42.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:42 smithi167 ceph-mon[29496]: osdmap e354: 8 total, 8 up, 8 in 2024-06-18T13:09:42.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi167-73991-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:43.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:43 smithi179 ceph-mon[33312]: osdmap e355: 8 total, 8 up, 8 in 2024-06-18T13:09:43.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:43 smithi179 ceph-mon[33312]: pgmap v485: 260 pgs: 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:09:43.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:43 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:43.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi167-73991-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi167-73991-65"}]': finished 2024-06-18T13:09:43.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:43 smithi179 ceph-mon[33312]: osdmap e356: 8 total, 8 up, 8 in 2024-06-18T13:09:43.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:43 smithi167 ceph-mon[37701]: osdmap e355: 8 total, 8 up, 8 in 2024-06-18T13:09:43.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:43 smithi167 ceph-mon[37701]: pgmap v485: 260 pgs: 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:09:43.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:43 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:43.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi167-73991-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi167-73991-65"}]': finished 2024-06-18T13:09:43.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:43 smithi167 ceph-mon[37701]: osdmap e356: 8 total, 8 up, 8 in 2024-06-18T13:09:43.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:43 smithi167 ceph-mon[29496]: osdmap e355: 8 total, 8 up, 8 in 2024-06-18T13:09:43.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:43 smithi167 ceph-mon[29496]: pgmap v485: 260 pgs: 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:09:43.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:43 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:43.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi167-73991-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi167-73991-65"}]': finished 2024-06-18T13:09:43.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:43 smithi167 ceph-mon[29496]: osdmap e356: 8 total, 8 up, 8 in 2024-06-18T13:09:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:44 smithi179 ceph-mon[33312]: osdmap e357: 8 total, 8 up, 8 in 2024-06-18T13:09:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:44 smithi179 ceph-mon[33312]: pgmap v488: 268 pgs: 8 creating+peering, 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:45.227 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:09:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:09:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:09:45.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:44 smithi167 ceph-mon[37701]: osdmap e357: 8 total, 8 up, 8 in 2024-06-18T13:09:45.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:44 smithi167 ceph-mon[37701]: pgmap v488: 268 pgs: 8 creating+peering, 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:45.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:44 smithi167 ceph-mon[29496]: osdmap e357: 8 total, 8 up, 8 in 2024-06-18T13:09:45.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:44 smithi167 ceph-mon[29496]: pgmap v488: 268 pgs: 8 creating+peering, 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:46.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:45 smithi179 ceph-mon[33312]: osdmap e358: 8 total, 8 up, 8 in 2024-06-18T13:09:46.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:46.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:45 smithi167 ceph-mon[29496]: osdmap e358: 8 total, 8 up, 8 in 2024-06-18T13:09:46.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:46.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:45 smithi167 ceph-mon[37701]: osdmap e358: 8 total, 8 up, 8 in 2024-06-18T13:09:46.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:47.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi167-73991-65"}]': finished 2024-06-18T13:09:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:46 smithi179 ceph-mon[33312]: osdmap e359: 8 total, 8 up, 8 in 2024-06-18T13:09:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:46 smithi179 ceph-mon[33312]: pgmap v491: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi167-73991-65"}]': finished 2024-06-18T13:09:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:09:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:46 smithi179 ceph-mon[33312]: osdmap e360: 8 total, 8 up, 8 in 2024-06-18T13:09:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:47.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi167-73991-65"}]': finished 2024-06-18T13:09:47.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[29496]: osdmap e359: 8 total, 8 up, 8 in 2024-06-18T13:09:47.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:47.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:47.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:47.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[29496]: pgmap v491: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:47.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi167-73991-65"}]': finished 2024-06-18T13:09:47.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:09:47.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[29496]: osdmap e360: 8 total, 8 up, 8 in 2024-06-18T13:09:47.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:47.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi167-73991-65"}]': finished 2024-06-18T13:09:47.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[37701]: osdmap e359: 8 total, 8 up, 8 in 2024-06-18T13:09:47.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi167-73991-65"}]: dispatch 2024-06-18T13:09:47.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:47.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:47.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[37701]: pgmap v491: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:47.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3181287040' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi167-73991-65"}]': finished 2024-06-18T13:09:47.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:09:47.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[37701]: osdmap e360: 8 total, 8 up, 8 in 2024-06-18T13:09:47.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi167-73991-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi167-73991-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:47 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:09:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]': finished 2024-06-18T13:09:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi167-73991-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:48.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:47 smithi179 ceph-mon[33312]: osdmap e361: 8 total, 8 up, 8 in 2024-06-18T13:09:48.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi167-73991-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi167-73991-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:48.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi167-73991-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:48.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi167-73991-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:48.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:09:48.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]': finished 2024-06-18T13:09:48.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi167-73991-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:48.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[29496]: osdmap e361: 8 total, 8 up, 8 in 2024-06-18T13:09:48.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi167-73991-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi167-73991-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:48.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi167-73991-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:48.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi167-73991-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:09:48.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:09:48.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]': finished 2024-06-18T13:09:48.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi167-73991-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:09:48.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[37701]: osdmap e361: 8 total, 8 up, 8 in 2024-06-18T13:09:48.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi167-73991-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:48.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi167-73991-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:49.075 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP:163:head 2024-06-18T13:09:49.075 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:5d165639:::164:head 2024-06-18T13:09:49.075 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:f43765fc:::165:head 2024-06-18T13:09:49.075 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:b4c720e9:::166:head 2024-06-18T13:09:49.075 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:e694b040:::167:head 2024-06-18T13:09:49.075 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:afa38db2:::168:head 2024-06-18T13:09:49.076 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:77ba9f53:::169:head 2024-06-18T13:09:49.076 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:87495034:::170:head 2024-06-18T13:09:49.076 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:7c96bf0e:::171:head 2024-06-18T13:09:49.076 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:dbe346cc:::172:head 2024-06-18T13:09:49.076 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:e943ec24:::173:head 2024-06-18T13:09:49.076 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:f97a9c0c:::174:head 2024-06-18T13:09:49.076 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:6f26e74d:::175:head 2024-06-18T13:09:49.076 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:4f95e106:::176:head 2024-06-18T13:09:49.076 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:0e6f2f8f:::177:head 2024-06-18T13:09:49.076 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:05db05f1:::178:head 2024-06-18T13:09:49.076 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:38a78d66:::179:head 2024-06-18T13:09:49.076 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:d095610b:::180:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:a1a9d709:::181:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:1e5d39db:::182:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:f7df4fb9:::183:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:03a7f161:::184:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:ba70721e:::185:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:28e5662d:::186:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:973d52de:::187:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:4303eb1c:::188:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:b990b48e:::189:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:29b8165b:::190:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:3547f197:::191:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:7e260936:::192:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:1abec7b1:::193:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:10fdda93:::194:head 2024-06-18T13:09:49.077 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:15817eea:::195:head 2024-06-18T13:09:49.078 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:770bab57:::196:head 2024-06-18T13:09:49.078 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:ed9e13e7:::197:head 2024-06-18T13:09:49.078 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:71471a8f:::198:head 2024-06-18T13:09:49.078 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: checking for 276:10fb1d02:::199:head 2024-06-18T13:09:49.078 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8277 ms) 2024-06-18T13:09:49.078 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-06-18T13:09:49.078 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716144,0 2024-06-18T13:09:49.078 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: first is 1718716144 2024-06-18T13:09:49.078 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716144,0 2024-06-18T13:09:49.078 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716144,0 2024-06-18T13:09:49.078 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716144,1718716146,1718716147,0 2024-06-18T13:09:49.078 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716144,1718716146,1718716147,0 2024-06-18T13:09:49.079 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716144,1718716146,1718716147,0 2024-06-18T13:09:49.079 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716144,1718716146,1718716147,1718716149,1718716150,0 2024-06-18T13:09:49.079 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716144,1718716146,1718716147,1718716149,1718716150,0 2024-06-18T13:09:49.079 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716144,1718716146,1718716147,1718716149,1718716150,0 2024-06-18T13:09:49.079 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716144,1718716146,1718716147,1718716149,1718716150,1718716152,1718716153,0 2024-06-18T13:09:49.079 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716144,1718716146,1718716147,1718716149,1718716150,1718716152,1718716153,0 2024-06-18T13:09:49.079 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716144,1718716146,1718716147,1718716149,1718716150,1718716152,1718716153,0 2024-06-18T13:09:49.079 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716147,1718716149,1718716150,1718716152,1718716153,1718716155,1718716156,0 2024-06-18T13:09:49.079 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: first now 1718716147, trimmed 2024-06-18T13:09:49.079 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (19906 ms) 2024-06-18T13:09:49.079 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-06-18T13:09:49.079 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: foo0 2024-06-18T13:09:49.079 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-06-18T13:09:49.080 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14087 ms) 2024-06-18T13:09:49.080 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-06-18T13:09:49.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:09:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:09:49.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:49 smithi179 ceph-mon[33312]: pgmap v494: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:49 smithi167 ceph-mon[29496]: pgmap v494: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:09:49.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:49.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-49"}]: dispatch 2024-06-18T13:09:49.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:49 smithi167 ceph-mon[37701]: pgmap v494: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:50 smithi179 ceph-mon[33312]: osdmap e362: 8 total, 8 up, 8 in 2024-06-18T13:09:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi167-73991-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi167-73991-66"}]': finished 2024-06-18T13:09:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:50 smithi179 ceph-mon[33312]: osdmap e363: 8 total, 8 up, 8 in 2024-06-18T13:09:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:50 smithi167 ceph-mon[29496]: osdmap e362: 8 total, 8 up, 8 in 2024-06-18T13:09:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi167-73991-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi167-73991-66"}]': finished 2024-06-18T13:09:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:50 smithi167 ceph-mon[29496]: osdmap e363: 8 total, 8 up, 8 in 2024-06-18T13:09:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:09:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:50 smithi167 ceph-mon[37701]: osdmap e362: 8 total, 8 up, 8 in 2024-06-18T13:09:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi167-73991-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi167-73991-66"}]': finished 2024-06-18T13:09:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:50 smithi167 ceph-mon[37701]: osdmap e363: 8 total, 8 up, 8 in 2024-06-18T13:09:50.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:50.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:09:51.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:51 smithi179 ceph-mon[33312]: pgmap v497: 268 pgs: 40 unknown, 228 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:51.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:51 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:51.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:51 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:09:51.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:51 smithi179 ceph-mon[33312]: osdmap e364: 8 total, 8 up, 8 in 2024-06-18T13:09:51.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:51 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:51.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:51 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:51 smithi167 ceph-mon[29496]: pgmap v497: 268 pgs: 40 unknown, 228 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:51 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:51 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:09:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:51 smithi167 ceph-mon[29496]: osdmap e364: 8 total, 8 up, 8 in 2024-06-18T13:09:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:51 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:51 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:51 smithi167 ceph-mon[37701]: pgmap v497: 268 pgs: 40 unknown, 228 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:51 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:51.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:51 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:09:51.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:51 smithi167 ceph-mon[37701]: osdmap e364: 8 total, 8 up, 8 in 2024-06-18T13:09:51.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:51 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:51.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:51 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:09:53.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:52 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:09:53.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:52 smithi179 ceph-mon[33312]: osdmap e365: 8 total, 8 up, 8 in 2024-06-18T13:09:53.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:52 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:09:53.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:52 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:09:53.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:52 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:53.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:52 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:53.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:52 smithi179 ceph-mon[33312]: pgmap v500: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:53.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:52 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:09:53.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:52 smithi167 ceph-mon[37701]: osdmap e365: 8 total, 8 up, 8 in 2024-06-18T13:09:53.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:52 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:09:53.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:52 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:09:53.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:52 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:53.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:52 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:53.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:52 smithi167 ceph-mon[37701]: pgmap v500: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:53.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:52 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:09:53.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:52 smithi167 ceph-mon[29496]: osdmap e365: 8 total, 8 up, 8 in 2024-06-18T13:09:53.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:52 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:09:53.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:52 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:09:53.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:52 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:53.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:52 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:53.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:52 smithi167 ceph-mon[29496]: pgmap v500: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:09:53.869 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ 2 2024-06-18T13:09:53.869 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3032 ms) 2024-06-18T13:09:53.869 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-06-18T13:09:53.869 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7070 ms) 2024-06-18T13:09:53.870 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-06-18T13:09:53.870 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7054 ms) 2024-06-18T13:09:53.870 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-06-18T13:09:53.870 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7028 ms) 2024-06-18T13:09:53.870 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-06-18T13:09:53.870 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7023 ms) 2024-06-18T13:09:53.870 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-06-18T13:09:53.870 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7033 ms) 2024-06-18T13:09:53.870 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-06-18T13:09:53.870 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7026 ms) 2024-06-18T13:09:53.870 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139438 ms total) 2024-06-18T13:09:53.870 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: 2024-06-18T13:09:53.870 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-06-18T13:09:53.871 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (299827 ms total) 2024-06-18T13:09:53.871 INFO:tasks.workunit.client.0.smithi167.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-06-18T13:09:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-51"}]': finished 2024-06-18T13:09:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]': finished 2024-06-18T13:09:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:53 smithi179 ceph-mon[33312]: osdmap e366: 8 total, 8 up, 8 in 2024-06-18T13:09:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-51", "mode": "writeback"}]: dispatch 2024-06-18T13:09:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-51", "mode": "writeback"}]: dispatch 2024-06-18T13:09:54.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-51"}]': finished 2024-06-18T13:09:54.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]': finished 2024-06-18T13:09:54.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:53 smithi167 ceph-mon[37701]: osdmap e366: 8 total, 8 up, 8 in 2024-06-18T13:09:54.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:54.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-51", "mode": "writeback"}]: dispatch 2024-06-18T13:09:54.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:54.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-51", "mode": "writeback"}]: dispatch 2024-06-18T13:09:54.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-51"}]': finished 2024-06-18T13:09:54.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi167-73991-66"}]': finished 2024-06-18T13:09:54.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:53 smithi167 ceph-mon[29496]: osdmap e366: 8 total, 8 up, 8 in 2024-06-18T13:09:54.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2357124126' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:54.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-51", "mode": "writeback"}]: dispatch 2024-06-18T13:09:54.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]: dispatch 2024-06-18T13:09:54.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-51", "mode": "writeback"}]: dispatch 2024-06-18T13:09:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:54 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:09:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]': finished 2024-06-18T13:09:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-51", "mode": "writeback"}]': finished 2024-06-18T13:09:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:54 smithi179 ceph-mon[33312]: osdmap e367: 8 total, 8 up, 8 in 2024-06-18T13:09:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:09:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:09:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:54 smithi179 ceph-mon[33312]: pgmap v503: 260 pgs: 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-18T13:09:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:54 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:09:55.226 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:09:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:09:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:09:55.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:09:55.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]': finished 2024-06-18T13:09:55.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-51", "mode": "writeback"}]': finished 2024-06-18T13:09:55.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[37701]: osdmap e367: 8 total, 8 up, 8 in 2024-06-18T13:09:55.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:09:55.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:09:55.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[37701]: pgmap v503: 260 pgs: 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-18T13:09:55.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:09:55.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:09:55.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi167-73991-66"}]': finished 2024-06-18T13:09:55.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-51", "mode": "writeback"}]': finished 2024-06-18T13:09:55.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[29496]: osdmap e367: 8 total, 8 up, 8 in 2024-06-18T13:09:55.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:09:55.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:09:55.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[29496]: pgmap v503: 260 pgs: 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-18T13:09:55.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:54 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:09:56.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:09:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:55 smithi179 ceph-mon[33312]: osdmap e368: 8 total, 8 up, 8 in 2024-06-18T13:09:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:09:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:09:56.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:09:56.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:55 smithi167 ceph-mon[37701]: osdmap e368: 8 total, 8 up, 8 in 2024-06-18T13:09:56.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:09:56.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:09:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:09:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:55 smithi167 ceph-mon[29496]: osdmap e368: 8 total, 8 up, 8 in 2024-06-18T13:09:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:09:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:09:57.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:09:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:56 smithi179 ceph-mon[33312]: osdmap e369: 8 total, 8 up, 8 in 2024-06-18T13:09:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:56 smithi179 ceph-mon[33312]: pgmap v506: 260 pgs: 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-18T13:09:57.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:09:57.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:56 smithi167 ceph-mon[37701]: osdmap e369: 8 total, 8 up, 8 in 2024-06-18T13:09:57.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:57.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:57.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:56 smithi167 ceph-mon[37701]: pgmap v506: 260 pgs: 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-18T13:09:57.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:09:57.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:56 smithi167 ceph-mon[29496]: osdmap e369: 8 total, 8 up, 8 in 2024-06-18T13:09:57.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:57.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:09:57.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:56 smithi167 ceph-mon[29496]: pgmap v506: 260 pgs: 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-18T13:09:58.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:57 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:09:58.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:09:58.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:57 smithi167 ceph-mon[37701]: osdmap e370: 8 total, 8 up, 8 in 2024-06-18T13:09:58.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:09:58.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:09:58.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:57 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:58.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:57 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:09:58.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:09:58.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:57 smithi167 ceph-mon[29496]: osdmap e370: 8 total, 8 up, 8 in 2024-06-18T13:09:58.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:09:58.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:09:58.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:57 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:58.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:57 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:09:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:09:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:57 smithi179 ceph-mon[33312]: osdmap e370: 8 total, 8 up, 8 in 2024-06-18T13:09:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:09:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:09:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:57 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:09:59.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-18T13:09:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:58 smithi179 ceph-mon[33312]: osdmap e371: 8 total, 8 up, 8 in 2024-06-18T13:09:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "target_max_objects","val": "1"}]: dispatch 2024-06-18T13:09:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "target_max_objects","val": "1"}]: dispatch 2024-06-18T13:09:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:58 smithi179 ceph-mon[33312]: pgmap v509: 260 pgs: 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "target_max_objects","val": "1"}]': finished 2024-06-18T13:09:59.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:09:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:09:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-18T13:09:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:58 smithi167 ceph-mon[37701]: osdmap e371: 8 total, 8 up, 8 in 2024-06-18T13:09:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "target_max_objects","val": "1"}]: dispatch 2024-06-18T13:09:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "target_max_objects","val": "1"}]: dispatch 2024-06-18T13:09:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:58 smithi167 ceph-mon[37701]: pgmap v509: 260 pgs: 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "target_max_objects","val": "1"}]': finished 2024-06-18T13:09:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-18T13:09:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:58 smithi167 ceph-mon[29496]: osdmap e371: 8 total, 8 up, 8 in 2024-06-18T13:09:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "target_max_objects","val": "1"}]: dispatch 2024-06-18T13:09:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "target_max_objects","val": "1"}]: dispatch 2024-06-18T13:09:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:58 smithi167 ceph-mon[29496]: pgmap v509: 260 pgs: 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:09:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-51","var": "target_max_objects","val": "1"}]': finished 2024-06-18T13:10:00.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:59 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:00.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:09:59 smithi167 ceph-mon[29496]: osdmap e372: 8 total, 8 up, 8 in 2024-06-18T13:10:00.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:59 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:00.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:09:59 smithi167 ceph-mon[37701]: osdmap e372: 8 total, 8 up, 8 in 2024-06-18T13:10:00.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:59 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:00.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:09:59 smithi179 ceph-mon[33312]: osdmap e372: 8 total, 8 up, 8 in 2024-06-18T13:10:01.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:00 smithi167 ceph-mon[37701]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-06-18T13:10:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:00 smithi167 ceph-mon[37701]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-06-18T13:10:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:00 smithi167 ceph-mon[37701]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-06-18T13:10:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:00 smithi167 ceph-mon[37701]: application not enabled on pool 'ceph_test_rados_delete_pools_parallel.smithi167-74857' 2024-06-18T13:10:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:00 smithi167 ceph-mon[37701]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-18T13:10:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:00 smithi167 ceph-mon[37701]: pgmap v511: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-18T13:10:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:00 smithi167 ceph-mon[29496]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-06-18T13:10:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:00 smithi167 ceph-mon[29496]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-06-18T13:10:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:00 smithi167 ceph-mon[29496]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-06-18T13:10:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:00 smithi167 ceph-mon[29496]: application not enabled on pool 'ceph_test_rados_delete_pools_parallel.smithi167-74857' 2024-06-18T13:10:01.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:00 smithi167 ceph-mon[29496]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-18T13:10:01.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:00 smithi167 ceph-mon[29496]: pgmap v511: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-18T13:10:01.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:00 smithi179 ceph-mon[33312]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-06-18T13:10:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:00 smithi179 ceph-mon[33312]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-06-18T13:10:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:00 smithi179 ceph-mon[33312]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-06-18T13:10:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:00 smithi179 ceph-mon[33312]: application not enabled on pool 'ceph_test_rados_delete_pools_parallel.smithi167-74857' 2024-06-18T13:10:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:00 smithi179 ceph-mon[33312]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-18T13:10:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:00 smithi179 ceph-mon[33312]: pgmap v511: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-18T13:10:02.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:01 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-06-18T13:10:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:01 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-06-18T13:10:02.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:01 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-06-18T13:10:03.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:02 smithi179 ceph-mon[33312]: pgmap v512: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:10:03.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:02 smithi167 ceph-mon[37701]: pgmap v512: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:10:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:02 smithi167 ceph-mon[29496]: pgmap v512: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:10:05.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:05 smithi167 ceph-mon[29496]: pgmap v513: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-18T13:10:05.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:05 smithi167 ceph-mon[37701]: pgmap v513: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-18T13:10:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:10:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:10:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:10:05.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:05 smithi179 ceph-mon[33312]: pgmap v513: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-18T13:10:07.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:07 smithi179 ceph-mon[33312]: pgmap v514: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:10:07.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:07 smithi167 ceph-mon[37701]: pgmap v514: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:10:07.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:07 smithi167 ceph-mon[29496]: pgmap v514: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:10:09.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:10:08 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:10:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:09 smithi179 ceph-mon[33312]: pgmap v515: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:10:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:09.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:09 smithi167 ceph-mon[37701]: pgmap v515: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:10:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:09 smithi167 ceph-mon[29496]: pgmap v515: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:10:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:10:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:10 smithi179 ceph-mon[33312]: osdmap e373: 8 total, 8 up, 8 in 2024-06-18T13:10:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:10:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:10:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:10:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:10:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]': finished 2024-06-18T13:10:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:10 smithi179 ceph-mon[33312]: osdmap e374: 8 total, 8 up, 8 in 2024-06-18T13:10:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:10:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[37701]: osdmap e373: 8 total, 8 up, 8 in 2024-06-18T13:10:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:10:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:10:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:10:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:10:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]': finished 2024-06-18T13:10:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[37701]: osdmap e374: 8 total, 8 up, 8 in 2024-06-18T13:10:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:10:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[29496]: osdmap e373: 8 total, 8 up, 8 in 2024-06-18T13:10:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:10:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:10:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:10:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:10:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]': finished 2024-06-18T13:10:10.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:10 smithi167 ceph-mon[29496]: osdmap e374: 8 total, 8 up, 8 in 2024-06-18T13:10:11.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:11 smithi179 ceph-mon[33312]: pgmap v517: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:10:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:10:11.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:11 smithi167 ceph-mon[37701]: pgmap v517: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:10:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:10:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:11 smithi167 ceph-mon[29496]: pgmap v517: 260 pgs: 260 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:10:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-51"}]: dispatch 2024-06-18T13:10:12.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:12 smithi179 ceph-mon[33312]: osdmap e375: 8 total, 8 up, 8 in 2024-06-18T13:10:12.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:12 smithi167 ceph-mon[37701]: osdmap e375: 8 total, 8 up, 8 in 2024-06-18T13:10:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:12 smithi167 ceph-mon[29496]: osdmap e375: 8 total, 8 up, 8 in 2024-06-18T13:10:13.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:13 smithi179 ceph-mon[33312]: pgmap v520: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-06-18T13:10:13.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:13 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-06-18T13:10:13.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:13 smithi179 ceph-mon[33312]: osdmap e376: 8 total, 8 up, 8 in 2024-06-18T13:10:13.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:13.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:13 smithi167 ceph-mon[37701]: pgmap v520: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-06-18T13:10:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:13 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-06-18T13:10:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:13 smithi167 ceph-mon[37701]: osdmap e376: 8 total, 8 up, 8 in 2024-06-18T13:10:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:13 smithi167 ceph-mon[29496]: pgmap v520: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-06-18T13:10:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:13 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-06-18T13:10:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:13 smithi167 ceph-mon[29496]: osdmap e376: 8 total, 8 up, 8 in 2024-06-18T13:10:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:13.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:14 smithi179 ceph-mon[33312]: osdmap e377: 8 total, 8 up, 8 in 2024-06-18T13:10:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-53"}]: dispatch 2024-06-18T13:10:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-53"}]: dispatch 2024-06-18T13:10:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:14 smithi167 ceph-mon[29496]: osdmap e377: 8 total, 8 up, 8 in 2024-06-18T13:10:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-53"}]: dispatch 2024-06-18T13:10:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-53"}]: dispatch 2024-06-18T13:10:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:14 smithi167 ceph-mon[37701]: osdmap e377: 8 total, 8 up, 8 in 2024-06-18T13:10:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-53"}]: dispatch 2024-06-18T13:10:14.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-53"}]: dispatch 2024-06-18T13:10:15.300 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:10:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:10:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:10:15.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:15 smithi179 ceph-mon[33312]: pgmap v523: 260 pgs: 13 creating+activating, 247 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-18T13:10:15.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:15 smithi179 ceph-mon[33312]: osdmap e378: 8 total, 8 up, 8 in 2024-06-18T13:10:15.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:15 smithi167 ceph-mon[29496]: pgmap v523: 260 pgs: 13 creating+activating, 247 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-18T13:10:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:15 smithi167 ceph-mon[29496]: osdmap e378: 8 total, 8 up, 8 in 2024-06-18T13:10:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:15 smithi167 ceph-mon[37701]: pgmap v523: 260 pgs: 13 creating+activating, 247 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-18T13:10:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:15 smithi167 ceph-mon[37701]: osdmap e378: 8 total, 8 up, 8 in 2024-06-18T13:10:16.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:16 smithi167 ceph-mon[37701]: osdmap e379: 8 total, 8 up, 8 in 2024-06-18T13:10:16.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:16.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:16.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:16.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:16 smithi167 ceph-mon[37701]: osdmap e380: 8 total, 8 up, 8 in 2024-06-18T13:10:16.579 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:16 smithi167 ceph-mon[29496]: osdmap e379: 8 total, 8 up, 8 in 2024-06-18T13:10:16.579 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:16.580 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:16.580 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:16.580 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:16 smithi167 ceph-mon[29496]: osdmap e380: 8 total, 8 up, 8 in 2024-06-18T13:10:16.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:16 smithi179 ceph-mon[33312]: osdmap e379: 8 total, 8 up, 8 in 2024-06-18T13:10:16.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:16 smithi179 ceph-mon[33312]: osdmap e380: 8 total, 8 up, 8 in 2024-06-18T13:10:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:17 smithi179 ceph-mon[33312]: pgmap v526: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:10:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:10:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:17 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:10:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-55"}]: dispatch 2024-06-18T13:10:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-55"}]: dispatch 2024-06-18T13:10:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:17 smithi179 ceph-mon[33312]: osdmap e381: 8 total, 8 up, 8 in 2024-06-18T13:10:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[37701]: pgmap v526: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:10:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:10:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:10:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-55"}]: dispatch 2024-06-18T13:10:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-55"}]: dispatch 2024-06-18T13:10:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[37701]: osdmap e381: 8 total, 8 up, 8 in 2024-06-18T13:10:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[29496]: pgmap v526: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:10:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:10:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:10:17.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:17.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:17.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-55"}]: dispatch 2024-06-18T13:10:17.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-55"}]: dispatch 2024-06-18T13:10:17.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:17 smithi167 ceph-mon[29496]: osdmap e381: 8 total, 8 up, 8 in 2024-06-18T13:10:19.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:10:18 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:10:19.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:19 smithi179 ceph-mon[33312]: pgmap v529: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:10:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:19 smithi179 ceph-mon[33312]: osdmap e382: 8 total, 8 up, 8 in 2024-06-18T13:10:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:19 smithi167 ceph-mon[37701]: pgmap v529: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:10:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:19 smithi167 ceph-mon[37701]: osdmap e382: 8 total, 8 up, 8 in 2024-06-18T13:10:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:19 smithi167 ceph-mon[29496]: pgmap v529: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:10:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:19 smithi167 ceph-mon[29496]: osdmap e382: 8 total, 8 up, 8 in 2024-06-18T13:10:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:20.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:20 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:20 smithi179 ceph-mon[33312]: osdmap e383: 8 total, 8 up, 8 in 2024-06-18T13:10:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-57"}]: dispatch 2024-06-18T13:10:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-57"}]: dispatch 2024-06-18T13:10:20.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:20 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:20 smithi167 ceph-mon[37701]: osdmap e383: 8 total, 8 up, 8 in 2024-06-18T13:10:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-57"}]: dispatch 2024-06-18T13:10:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-57"}]: dispatch 2024-06-18T13:10:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:20 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:20 smithi167 ceph-mon[29496]: osdmap e383: 8 total, 8 up, 8 in 2024-06-18T13:10:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-57"}]: dispatch 2024-06-18T13:10:20.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-57"}]: dispatch 2024-06-18T13:10:21.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:21 smithi179 ceph-mon[33312]: pgmap v532: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:10:21.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:21 smithi179 ceph-mon[33312]: osdmap e384: 8 total, 8 up, 8 in 2024-06-18T13:10:21.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:21 smithi167 ceph-mon[37701]: pgmap v532: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:10:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:21 smithi167 ceph-mon[37701]: osdmap e384: 8 total, 8 up, 8 in 2024-06-18T13:10:21.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:21 smithi167 ceph-mon[29496]: pgmap v532: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:10:21.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:21 smithi167 ceph-mon[29496]: osdmap e384: 8 total, 8 up, 8 in 2024-06-18T13:10:22.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:22 smithi179 ceph-mon[33312]: osdmap e385: 8 total, 8 up, 8 in 2024-06-18T13:10:22.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:22.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:22 smithi167 ceph-mon[37701]: osdmap e385: 8 total, 8 up, 8 in 2024-06-18T13:10:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:22.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:22 smithi167 ceph-mon[29496]: osdmap e385: 8 total, 8 up, 8 in 2024-06-18T13:10:22.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:22.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:23 smithi179 ceph-mon[33312]: pgmap v535: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:10:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:23 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:10:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:23 smithi179 ceph-mon[33312]: osdmap e386: 8 total, 8 up, 8 in 2024-06-18T13:10:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:10:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-59"}]: dispatch 2024-06-18T13:10:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-59"}]: dispatch 2024-06-18T13:10:23.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[37701]: pgmap v535: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:10:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:10:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[37701]: osdmap e386: 8 total, 8 up, 8 in 2024-06-18T13:10:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:10:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-59"}]: dispatch 2024-06-18T13:10:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-59"}]: dispatch 2024-06-18T13:10:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[29496]: pgmap v535: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:10:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:10:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[29496]: osdmap e386: 8 total, 8 up, 8 in 2024-06-18T13:10:23.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:10:23.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:23.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:23.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-59"}]: dispatch 2024-06-18T13:10:23.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-59"}]: dispatch 2024-06-18T13:10:24.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:24 smithi179 ceph-mon[33312]: osdmap e387: 8 total, 8 up, 8 in 2024-06-18T13:10:24.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:24 smithi167 ceph-mon[29496]: osdmap e387: 8 total, 8 up, 8 in 2024-06-18T13:10:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:24 smithi167 ceph-mon[37701]: osdmap e387: 8 total, 8 up, 8 in 2024-06-18T13:10:25.356 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:10:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:10:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:10:25.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:25 smithi179 ceph-mon[33312]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:10:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:25 smithi179 ceph-mon[33312]: osdmap e388: 8 total, 8 up, 8 in 2024-06-18T13:10:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:10:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:10:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:25 smithi167 ceph-mon[37701]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:10:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:25 smithi167 ceph-mon[37701]: osdmap e388: 8 total, 8 up, 8 in 2024-06-18T13:10:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:10:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:10:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:25 smithi167 ceph-mon[29496]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:10:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:25 smithi167 ceph-mon[29496]: osdmap e388: 8 total, 8 up, 8 in 2024-06-18T13:10:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:25.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:25.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:10:25.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:10:26.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:26.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:26 smithi179 ceph-mon[33312]: osdmap e389: 8 total, 8 up, 8 in 2024-06-18T13:10:26.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:10:26.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:10:26.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:10:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:26 smithi167 ceph-mon[37701]: osdmap e389: 8 total, 8 up, 8 in 2024-06-18T13:10:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:10:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:10:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:10:26.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:26.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:26 smithi167 ceph-mon[29496]: osdmap e389: 8 total, 8 up, 8 in 2024-06-18T13:10:26.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:10:26.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:10:26.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:10:27.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:27 smithi179 ceph-mon[33312]: pgmap v541: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:10:27.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:10:27.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:27 smithi179 ceph-mon[33312]: osdmap e390: 8 total, 8 up, 8 in 2024-06-18T13:10:27.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:27.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:27.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-61"}]: dispatch 2024-06-18T13:10:27.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-61"}]: dispatch 2024-06-18T13:10:27.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:27 smithi167 ceph-mon[37701]: pgmap v541: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:10:27.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:10:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:27 smithi167 ceph-mon[37701]: osdmap e390: 8 total, 8 up, 8 in 2024-06-18T13:10:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-61"}]: dispatch 2024-06-18T13:10:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-61"}]: dispatch 2024-06-18T13:10:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:27 smithi167 ceph-mon[29496]: pgmap v541: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:10:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:10:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:27 smithi167 ceph-mon[29496]: osdmap e390: 8 total, 8 up, 8 in 2024-06-18T13:10:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:10:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-61"}]: dispatch 2024-06-18T13:10:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-61"}]: dispatch 2024-06-18T13:10:28.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:28 smithi179 ceph-mon[33312]: osdmap e391: 8 total, 8 up, 8 in 2024-06-18T13:10:28.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:28 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:10:28.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:28 smithi167 ceph-mon[37701]: osdmap e391: 8 total, 8 up, 8 in 2024-06-18T13:10:28.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:28 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:10:28.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:28 smithi167 ceph-mon[29496]: osdmap e391: 8 total, 8 up, 8 in 2024-06-18T13:10:28.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:28 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:10:29.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:10:28 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:10:29.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:29 smithi179 ceph-mon[33312]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:10:29.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:29 smithi179 ceph-mon[33312]: osdmap e392: 8 total, 8 up, 8 in 2024-06-18T13:10:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:29.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:29 smithi167 ceph-mon[37701]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:10:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:29 smithi167 ceph-mon[37701]: osdmap e392: 8 total, 8 up, 8 in 2024-06-18T13:10:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:29 smithi167 ceph-mon[29496]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:10:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:29 smithi167 ceph-mon[29496]: osdmap e392: 8 total, 8 up, 8 in 2024-06-18T13:10:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:10:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:30 smithi167 ceph-mon[37701]: osdmap e393: 8 total, 8 up, 8 in 2024-06-18T13:10:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:10:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:10:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:10:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:30.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:30 smithi167 ceph-mon[29496]: osdmap e393: 8 total, 8 up, 8 in 2024-06-18T13:10:30.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:10:30.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:10:30.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:10:30.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:10:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:30 smithi179 ceph-mon[33312]: osdmap e393: 8 total, 8 up, 8 in 2024-06-18T13:10:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:10:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:10:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:10:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:31 smithi167 ceph-mon[37701]: pgmap v547: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:10:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:10:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:31 smithi167 ceph-mon[37701]: osdmap e394: 8 total, 8 up, 8 in 2024-06-18T13:10:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:31 smithi167 ceph-mon[29496]: pgmap v547: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:10:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:10:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:31 smithi167 ceph-mon[29496]: osdmap e394: 8 total, 8 up, 8 in 2024-06-18T13:10:31.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:31 smithi179 ceph-mon[33312]: pgmap v547: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:10:31.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:10:31.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:31 smithi179 ceph-mon[33312]: osdmap e394: 8 total, 8 up, 8 in 2024-06-18T13:10:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:32 smithi167 ceph-mon[37701]: osdmap e395: 8 total, 8 up, 8 in 2024-06-18T13:10:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:32 smithi167 ceph-mon[29496]: osdmap e395: 8 total, 8 up, 8 in 2024-06-18T13:10:32.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:32 smithi179 ceph-mon[33312]: osdmap e395: 8 total, 8 up, 8 in 2024-06-18T13:10:33.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:33 smithi167 ceph-mon[37701]: pgmap v550: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:10:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:33 smithi167 ceph-mon[37701]: osdmap e396: 8 total, 8 up, 8 in 2024-06-18T13:10:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:33 smithi167 ceph-mon[29496]: pgmap v550: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:10:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:33 smithi167 ceph-mon[29496]: osdmap e396: 8 total, 8 up, 8 in 2024-06-18T13:10:33.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:33 smithi179 ceph-mon[33312]: pgmap v550: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:10:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:33 smithi179 ceph-mon[33312]: osdmap e396: 8 total, 8 up, 8 in 2024-06-18T13:10:34.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:34 smithi167 ceph-mon[37701]: osdmap e397: 8 total, 8 up, 8 in 2024-06-18T13:10:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:34 smithi167 ceph-mon[29496]: osdmap e397: 8 total, 8 up, 8 in 2024-06-18T13:10:34.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:34 smithi179 ceph-mon[33312]: osdmap e397: 8 total, 8 up, 8 in 2024-06-18T13:10:35.420 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:10:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:10:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:10:35.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:35 smithi167 ceph-mon[37701]: pgmap v553: 260 pgs: 260 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-06-18T13:10:35.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:35 smithi167 ceph-mon[37701]: osdmap e398: 8 total, 8 up, 8 in 2024-06-18T13:10:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:35 smithi167 ceph-mon[29496]: pgmap v553: 260 pgs: 260 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-06-18T13:10:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:35 smithi167 ceph-mon[29496]: osdmap e398: 8 total, 8 up, 8 in 2024-06-18T13:10:35.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:35 smithi179 ceph-mon[33312]: pgmap v553: 260 pgs: 260 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-06-18T13:10:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:35 smithi179 ceph-mon[33312]: osdmap e398: 8 total, 8 up, 8 in 2024-06-18T13:10:37.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:37 smithi167 ceph-mon[37701]: pgmap v555: 260 pgs: 260 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-06-18T13:10:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:37 smithi167 ceph-mon[29496]: pgmap v555: 260 pgs: 260 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-06-18T13:10:37.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:37 smithi179 ceph-mon[33312]: pgmap v555: 260 pgs: 260 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-06-18T13:10:39.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:10:38 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:10:39.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:39 smithi167 ceph-mon[37701]: pgmap v556: 260 pgs: 260 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-06-18T13:10:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:39 smithi167 ceph-mon[29496]: pgmap v556: 260 pgs: 260 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-06-18T13:10:39.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:39 smithi179 ceph-mon[33312]: pgmap v556: 260 pgs: 260 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-06-18T13:10:40.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:40.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:10:40.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:10:40.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:10:40.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:10:40.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:10:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:10:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:10:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:10:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:10:40.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:10:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:10:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:10:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:10:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:10:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:10:41.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:41 smithi167 ceph-mon[37701]: pgmap v557: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-06-18T13:10:41.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:41 smithi167 ceph-mon[29496]: pgmap v557: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-06-18T13:10:41.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:41 smithi179 ceph-mon[33312]: pgmap v557: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-06-18T13:10:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:43 smithi167 ceph-mon[37701]: pgmap v558: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-06-18T13:10:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:43 smithi167 ceph-mon[37701]: osdmap e399: 8 total, 8 up, 8 in 2024-06-18T13:10:43.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:43 smithi167 ceph-mon[29496]: pgmap v558: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-06-18T13:10:43.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:43 smithi167 ceph-mon[29496]: osdmap e399: 8 total, 8 up, 8 in 2024-06-18T13:10:43.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:43 smithi179 ceph-mon[33312]: pgmap v558: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-06-18T13:10:43.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:43 smithi179 ceph-mon[33312]: osdmap e399: 8 total, 8 up, 8 in 2024-06-18T13:10:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:44 smithi179 ceph-mon[33312]: pgmap v560: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:44 smithi179 ceph-mon[33312]: osdmap e400: 8 total, 8 up, 8 in 2024-06-18T13:10:44.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:44 smithi167 ceph-mon[37701]: pgmap v560: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:44.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:44 smithi167 ceph-mon[37701]: osdmap e400: 8 total, 8 up, 8 in 2024-06-18T13:10:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:44 smithi167 ceph-mon[29496]: pgmap v560: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:44 smithi167 ceph-mon[29496]: osdmap e400: 8 total, 8 up, 8 in 2024-06-18T13:10:45.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:10:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:10:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:10:47.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:47 smithi179 ceph-mon[33312]: pgmap v562: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:47.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:47 smithi167 ceph-mon[29496]: pgmap v562: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:47 smithi167 ceph-mon[37701]: pgmap v562: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:49.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:10:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:10:49.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:49 smithi179 ceph-mon[33312]: pgmap v563: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:10:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:49 smithi167 ceph-mon[37701]: pgmap v563: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:10:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:49 smithi167 ceph-mon[29496]: pgmap v563: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:10:50.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:50.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:10:51.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:51 smithi179 ceph-mon[33312]: pgmap v564: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:51.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:51 smithi167 ceph-mon[29496]: pgmap v564: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:51 smithi167 ceph-mon[37701]: pgmap v564: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:53.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:53 smithi179 ceph-mon[33312]: pgmap v565: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:53.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:53 smithi179 ceph-mon[33312]: osdmap e401: 8 total, 8 up, 8 in 2024-06-18T13:10:53.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:53 smithi167 ceph-mon[37701]: pgmap v565: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:53.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:53 smithi167 ceph-mon[37701]: osdmap e401: 8 total, 8 up, 8 in 2024-06-18T13:10:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:53 smithi167 ceph-mon[29496]: pgmap v565: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:53 smithi167 ceph-mon[29496]: osdmap e401: 8 total, 8 up, 8 in 2024-06-18T13:10:55.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:55 smithi167 ceph-mon[29496]: pgmap v567: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:55.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:10:55.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:55 smithi167 ceph-mon[37701]: pgmap v567: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:55.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:10:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:10:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:10:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:10:55.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:55 smithi179 ceph-mon[33312]: pgmap v567: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:55.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:10:56.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:56 smithi179 ceph-mon[33312]: osdmap e402: 8 total, 8 up, 8 in 2024-06-18T13:10:56.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:56 smithi167 ceph-mon[29496]: osdmap e402: 8 total, 8 up, 8 in 2024-06-18T13:10:56.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:56 smithi167 ceph-mon[37701]: osdmap e402: 8 total, 8 up, 8 in 2024-06-18T13:10:57.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:57 smithi179 ceph-mon[33312]: pgmap v569: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:57.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:57 smithi167 ceph-mon[37701]: pgmap v569: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:57.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:57 smithi167 ceph-mon[29496]: pgmap v569: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:10:59.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:10:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:10:59.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:10:59 smithi179 ceph-mon[33312]: pgmap v570: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:10:59.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:10:59 smithi167 ceph-mon[37701]: pgmap v570: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:10:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:10:59 smithi167 ceph-mon[29496]: pgmap v570: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:11:00.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:01.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:01 smithi179 ceph-mon[33312]: pgmap v571: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-18T13:11:01.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:01 smithi167 ceph-mon[37701]: pgmap v571: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-18T13:11:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:01 smithi167 ceph-mon[29496]: pgmap v571: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-18T13:11:03.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:03 smithi179 ceph-mon[33312]: pgmap v572: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:11:03.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:03 smithi179 ceph-mon[33312]: osdmap e403: 8 total, 8 up, 8 in 2024-06-18T13:11:03.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:03 smithi167 ceph-mon[37701]: pgmap v572: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:11:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:03 smithi167 ceph-mon[37701]: osdmap e403: 8 total, 8 up, 8 in 2024-06-18T13:11:03.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:03 smithi167 ceph-mon[29496]: pgmap v572: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:11:03.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:03 smithi167 ceph-mon[29496]: osdmap e403: 8 total, 8 up, 8 in 2024-06-18T13:11:05.297 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:11:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:11:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:11:05.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:05 smithi179 ceph-mon[33312]: pgmap v574: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:11:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:05 smithi167 ceph-mon[29496]: pgmap v574: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:11:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:05 smithi167 ceph-mon[37701]: pgmap v574: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:11:06.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:06.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:06.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-63"}]: dispatch 2024-06-18T13:11:06.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-63"}]: dispatch 2024-06-18T13:11:06.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:06.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:06.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-63"}]: dispatch 2024-06-18T13:11:06.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-63"}]: dispatch 2024-06-18T13:11:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-63"}]: dispatch 2024-06-18T13:11:06.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-63"}]: dispatch 2024-06-18T13:11:07.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:07 smithi179 ceph-mon[33312]: pgmap v575: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:11:07.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:07 smithi179 ceph-mon[33312]: osdmap e404: 8 total, 8 up, 8 in 2024-06-18T13:11:07.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:07 smithi167 ceph-mon[29496]: pgmap v575: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:11:07.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:07 smithi167 ceph-mon[29496]: osdmap e404: 8 total, 8 up, 8 in 2024-06-18T13:11:07.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:07 smithi167 ceph-mon[37701]: pgmap v575: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:11:07.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:07 smithi167 ceph-mon[37701]: osdmap e404: 8 total, 8 up, 8 in 2024-06-18T13:11:08.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:08 smithi179 ceph-mon[33312]: osdmap e405: 8 total, 8 up, 8 in 2024-06-18T13:11:08.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:08.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:08.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:08 smithi167 ceph-mon[29496]: osdmap e405: 8 total, 8 up, 8 in 2024-06-18T13:11:08.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:08.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:08 smithi167 ceph-mon[37701]: osdmap e405: 8 total, 8 up, 8 in 2024-06-18T13:11:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:09.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:11:08 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:11:09.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:09 smithi179 ceph-mon[33312]: pgmap v578: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:09 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:11:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:09 smithi179 ceph-mon[33312]: osdmap e406: 8 total, 8 up, 8 in 2024-06-18T13:11:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:11:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:11:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:09 smithi167 ceph-mon[29496]: pgmap v578: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:09 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:11:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:09 smithi167 ceph-mon[29496]: osdmap e406: 8 total, 8 up, 8 in 2024-06-18T13:11:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:11:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:11:09.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:09 smithi167 ceph-mon[37701]: pgmap v578: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:09.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:09 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:11:09.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:09.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:09 smithi167 ceph-mon[37701]: osdmap e406: 8 total, 8 up, 8 in 2024-06-18T13:11:09.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:09.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:11:09.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:11:10.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:10.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:11:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:10 smithi179 ceph-mon[33312]: osdmap e407: 8 total, 8 up, 8 in 2024-06-18T13:11:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:11:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:11:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:10 smithi167 ceph-mon[37701]: osdmap e407: 8 total, 8 up, 8 in 2024-06-18T13:11:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:11:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:11:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:10 smithi167 ceph-mon[29496]: osdmap e407: 8 total, 8 up, 8 in 2024-06-18T13:11:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:11:11.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:11 smithi179 ceph-mon[33312]: pgmap v581: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:11 smithi179 ceph-mon[33312]: osdmap e408: 8 total, 8 up, 8 in 2024-06-18T13:11:11.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:11 smithi167 ceph-mon[29496]: pgmap v581: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:11 smithi167 ceph-mon[29496]: osdmap e408: 8 total, 8 up, 8 in 2024-06-18T13:11:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:11 smithi167 ceph-mon[37701]: pgmap v581: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:11 smithi167 ceph-mon[37701]: osdmap e408: 8 total, 8 up, 8 in 2024-06-18T13:11:12.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:12 smithi179 ceph-mon[33312]: osdmap e409: 8 total, 8 up, 8 in 2024-06-18T13:11:12.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:12 smithi167 ceph-mon[29496]: osdmap e409: 8 total, 8 up, 8 in 2024-06-18T13:11:12.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:12 smithi167 ceph-mon[37701]: osdmap e409: 8 total, 8 up, 8 in 2024-06-18T13:11:13.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:13 smithi179 ceph-mon[33312]: pgmap v584: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:11:13.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:13 smithi179 ceph-mon[33312]: osdmap e410: 8 total, 8 up, 8 in 2024-06-18T13:11:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:13 smithi167 ceph-mon[37701]: pgmap v584: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:11:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:13 smithi167 ceph-mon[37701]: osdmap e410: 8 total, 8 up, 8 in 2024-06-18T13:11:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:13 smithi167 ceph-mon[29496]: pgmap v584: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:11:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:13 smithi167 ceph-mon[29496]: osdmap e410: 8 total, 8 up, 8 in 2024-06-18T13:11:15.353 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:11:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:11:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:11:15.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:15 smithi179 ceph-mon[33312]: pgmap v586: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-18T13:11:15.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:15 smithi167 ceph-mon[37701]: pgmap v586: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-18T13:11:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:15 smithi167 ceph-mon[29496]: pgmap v586: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-18T13:11:17.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:17 smithi179 ceph-mon[33312]: pgmap v587: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-18T13:11:17.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:17 smithi167 ceph-mon[37701]: pgmap v587: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-18T13:11:17.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:17 smithi167 ceph-mon[29496]: pgmap v587: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-18T13:11:18.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:18 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:11:18.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:18 smithi179 ceph-mon[33312]: osdmap e411: 8 total, 8 up, 8 in 2024-06-18T13:11:18.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:18 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:11:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:18 smithi167 ceph-mon[37701]: osdmap e411: 8 total, 8 up, 8 in 2024-06-18T13:11:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:18 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:11:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:18 smithi167 ceph-mon[29496]: osdmap e411: 8 total, 8 up, 8 in 2024-06-18T13:11:19.361 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:11:18 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:11:19.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:19 smithi179 ceph-mon[33312]: pgmap v589: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-06-18T13:11:19.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:19 smithi167 ceph-mon[37701]: pgmap v589: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-06-18T13:11:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:19 smithi167 ceph-mon[29496]: pgmap v589: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-06-18T13:11:20.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:20 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:20.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:20 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:20 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:21.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:21 smithi179 ceph-mon[33312]: pgmap v590: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-06-18T13:11:21.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:21 smithi167 ceph-mon[37701]: pgmap v590: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-06-18T13:11:21.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:21 smithi167 ceph-mon[29496]: pgmap v590: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-06-18T13:11:23.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:23 smithi179 ceph-mon[33312]: pgmap v591: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-06-18T13:11:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-65"}]: dispatch 2024-06-18T13:11:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-65"}]: dispatch 2024-06-18T13:11:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:23 smithi167 ceph-mon[37701]: pgmap v591: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-06-18T13:11:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-65"}]: dispatch 2024-06-18T13:11:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-65"}]: dispatch 2024-06-18T13:11:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:23 smithi167 ceph-mon[29496]: pgmap v591: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-06-18T13:11:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-65"}]: dispatch 2024-06-18T13:11:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-65"}]: dispatch 2024-06-18T13:11:24.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:24 smithi179 ceph-mon[33312]: osdmap e412: 8 total, 8 up, 8 in 2024-06-18T13:11:24.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:24 smithi167 ceph-mon[29496]: osdmap e412: 8 total, 8 up, 8 in 2024-06-18T13:11:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:24 smithi167 ceph-mon[37701]: osdmap e412: 8 total, 8 up, 8 in 2024-06-18T13:11:25.384 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:11:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:11:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:11:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:25 smithi179 ceph-mon[33312]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:25 smithi179 ceph-mon[33312]: osdmap e413: 8 total, 8 up, 8 in 2024-06-18T13:11:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:25.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:11:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:25 smithi167 ceph-mon[29496]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:25 smithi167 ceph-mon[29496]: osdmap e413: 8 total, 8 up, 8 in 2024-06-18T13:11:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:11:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:25 smithi167 ceph-mon[37701]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:25 smithi167 ceph-mon[37701]: osdmap e413: 8 total, 8 up, 8 in 2024-06-18T13:11:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:25.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:25.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:11:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:26 smithi167 ceph-mon[37701]: osdmap e414: 8 total, 8 up, 8 in 2024-06-18T13:11:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:26 smithi167 ceph-mon[29496]: osdmap e414: 8 total, 8 up, 8 in 2024-06-18T13:11:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:26.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:26 smithi179 ceph-mon[33312]: osdmap e414: 8 total, 8 up, 8 in 2024-06-18T13:11:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:27.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:27 smithi167 ceph-mon[29496]: pgmap v596: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:27.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:27 smithi167 ceph-mon[29496]: osdmap e415: 8 total, 8 up, 8 in 2024-06-18T13:11:27.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:27.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:27.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-67"}]: dispatch 2024-06-18T13:11:27.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-67"}]: dispatch 2024-06-18T13:11:27.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:27 smithi167 ceph-mon[37701]: pgmap v596: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:27.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:27 smithi167 ceph-mon[37701]: osdmap e415: 8 total, 8 up, 8 in 2024-06-18T13:11:27.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:27.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:27.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-67"}]: dispatch 2024-06-18T13:11:27.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-67"}]: dispatch 2024-06-18T13:11:27.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:27 smithi179 ceph-mon[33312]: pgmap v596: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:27 smithi179 ceph-mon[33312]: osdmap e415: 8 total, 8 up, 8 in 2024-06-18T13:11:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-67"}]: dispatch 2024-06-18T13:11:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-67"}]: dispatch 2024-06-18T13:11:28.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:28 smithi167 ceph-mon[37701]: osdmap e416: 8 total, 8 up, 8 in 2024-06-18T13:11:28.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:28 smithi167 ceph-mon[29496]: osdmap e416: 8 total, 8 up, 8 in 2024-06-18T13:11:28.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:28 smithi179 ceph-mon[33312]: osdmap e416: 8 total, 8 up, 8 in 2024-06-18T13:11:29.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:11:28 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:11:29.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:29 smithi167 ceph-mon[37701]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:29 smithi167 ceph-mon[37701]: osdmap e417: 8 total, 8 up, 8 in 2024-06-18T13:11:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:29 smithi167 ceph-mon[29496]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:29 smithi167 ceph-mon[29496]: osdmap e417: 8 total, 8 up, 8 in 2024-06-18T13:11:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:29.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:29 smithi179 ceph-mon[33312]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:29 smithi179 ceph-mon[33312]: osdmap e417: 8 total, 8 up, 8 in 2024-06-18T13:11:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[37701]: osdmap e418: 8 total, 8 up, 8 in 2024-06-18T13:11:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-69"}]: dispatch 2024-06-18T13:11:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-69"}]: dispatch 2024-06-18T13:11:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:30.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[29496]: osdmap e418: 8 total, 8 up, 8 in 2024-06-18T13:11:30.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:30.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:30.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:30.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-69"}]: dispatch 2024-06-18T13:11:30.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-69"}]: dispatch 2024-06-18T13:11:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:30 smithi179 ceph-mon[33312]: osdmap e418: 8 total, 8 up, 8 in 2024-06-18T13:11:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-69"}]: dispatch 2024-06-18T13:11:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-69"}]: dispatch 2024-06-18T13:11:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:31 smithi167 ceph-mon[37701]: pgmap v602: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:11:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:31 smithi167 ceph-mon[37701]: osdmap e419: 8 total, 8 up, 8 in 2024-06-18T13:11:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:31 smithi167 ceph-mon[29496]: pgmap v602: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:11:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:31 smithi167 ceph-mon[29496]: osdmap e419: 8 total, 8 up, 8 in 2024-06-18T13:11:31.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:31 smithi179 ceph-mon[33312]: pgmap v602: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:11:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:31 smithi179 ceph-mon[33312]: osdmap e419: 8 total, 8 up, 8 in 2024-06-18T13:11:32.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:32 smithi167 ceph-mon[37701]: osdmap e420: 8 total, 8 up, 8 in 2024-06-18T13:11:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:32 smithi167 ceph-mon[37701]: osdmap e421: 8 total, 8 up, 8 in 2024-06-18T13:11:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:32 smithi167 ceph-mon[29496]: osdmap e420: 8 total, 8 up, 8 in 2024-06-18T13:11:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:32 smithi167 ceph-mon[29496]: osdmap e421: 8 total, 8 up, 8 in 2024-06-18T13:11:32.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:32 smithi179 ceph-mon[33312]: osdmap e420: 8 total, 8 up, 8 in 2024-06-18T13:11:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:32 smithi179 ceph-mon[33312]: osdmap e421: 8 total, 8 up, 8 in 2024-06-18T13:11:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:33 smithi167 ceph-mon[37701]: pgmap v605: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:11:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:33.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:33 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:11:33.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:11:33.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:33 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:11:33.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:33 smithi167 ceph-mon[29496]: pgmap v605: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:11:33.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:33.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:33 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:11:33.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:11:33.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:33 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:11:33.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:33 smithi179 ceph-mon[33312]: pgmap v605: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:11:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:33 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:11:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:11:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:33 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:11:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:11:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:34 smithi167 ceph-mon[37701]: osdmap e422: 8 total, 8 up, 8 in 2024-06-18T13:11:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:34 smithi167 ceph-mon[37701]: pgmap v608: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-06-18T13:11:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:11:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:34 smithi167 ceph-mon[29496]: osdmap e422: 8 total, 8 up, 8 in 2024-06-18T13:11:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:34 smithi167 ceph-mon[29496]: pgmap v608: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-06-18T13:11:34.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:11:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:34 smithi179 ceph-mon[33312]: osdmap e422: 8 total, 8 up, 8 in 2024-06-18T13:11:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:34 smithi179 ceph-mon[33312]: pgmap v608: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-06-18T13:11:35.469 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:11:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:11:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:11:35.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:35 smithi167 ceph-mon[37701]: osdmap e423: 8 total, 8 up, 8 in 2024-06-18T13:11:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:35 smithi167 ceph-mon[29496]: osdmap e423: 8 total, 8 up, 8 in 2024-06-18T13:11:35.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:35 smithi179 ceph-mon[33312]: osdmap e423: 8 total, 8 up, 8 in 2024-06-18T13:11:36.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:36 smithi179 ceph-mon[33312]: pgmap v610: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-06-18T13:11:36.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:36 smithi167 ceph-mon[37701]: pgmap v610: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-06-18T13:11:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:36 smithi167 ceph-mon[29496]: pgmap v610: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-06-18T13:11:37.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:37 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:11:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:37 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:11:38.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:37 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:11:38.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:38 smithi167 ceph-mon[37701]: pgmap v611: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-06-18T13:11:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:38 smithi167 ceph-mon[29496]: pgmap v611: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-06-18T13:11:39.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:38 smithi179 ceph-mon[33312]: pgmap v611: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-06-18T13:11:39.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:11:38 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:11:39.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:39 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:39.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:11:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:39 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:11:40.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:39 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:40.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:11:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:40 smithi167 ceph-mon[37701]: pgmap v612: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-06-18T13:11:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:11:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:40 smithi167 ceph-mon[29496]: pgmap v612: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-06-18T13:11:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:11:41.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:40 smithi179 ceph-mon[33312]: pgmap v612: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-06-18T13:11:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:11:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:41 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:11:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:41 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:11:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:41 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:11:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:41 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:11:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:41 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:11:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:41 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:11:42.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:41 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:11:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:41 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:11:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:41 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:11:43.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:42 smithi179 ceph-mon[33312]: pgmap v613: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-06-18T13:11:43.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:42 smithi167 ceph-mon[37701]: pgmap v613: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-06-18T13:11:43.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:42 smithi167 ceph-mon[29496]: pgmap v613: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-06-18T13:11:45.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:45 smithi167 ceph-mon[37701]: pgmap v614: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 409 B/s wr, 1 op/s 2024-06-18T13:11:45.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:45 smithi167 ceph-mon[29496]: pgmap v614: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 409 B/s wr, 1 op/s 2024-06-18T13:11:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:11:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:11:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:11:45.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:45 smithi179 ceph-mon[33312]: pgmap v614: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 409 B/s wr, 1 op/s 2024-06-18T13:11:47.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:47 smithi179 ceph-mon[33312]: pgmap v615: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 959 B/s rd, 348 B/s wr, 1 op/s 2024-06-18T13:11:47.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:47 smithi167 ceph-mon[37701]: pgmap v615: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 959 B/s rd, 348 B/s wr, 1 op/s 2024-06-18T13:11:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:47 smithi167 ceph-mon[29496]: pgmap v615: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 959 B/s rd, 348 B/s wr, 1 op/s 2024-06-18T13:11:49.277 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:11:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:11:49.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:49 smithi179 ceph-mon[33312]: pgmap v616: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-06-18T13:11:49.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:49 smithi167 ceph-mon[37701]: pgmap v616: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-06-18T13:11:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:49 smithi167 ceph-mon[29496]: pgmap v616: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-06-18T13:11:50.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:50.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:11:51.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:51 smithi179 ceph-mon[33312]: pgmap v617: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 597 B/s wr, 3 op/s 2024-06-18T13:11:51.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:51 smithi167 ceph-mon[37701]: pgmap v617: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 597 B/s wr, 3 op/s 2024-06-18T13:11:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:51 smithi167 ceph-mon[29496]: pgmap v617: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 597 B/s wr, 3 op/s 2024-06-18T13:11:53.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:53 smithi179 ceph-mon[33312]: pgmap v618: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:11:53.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:53 smithi167 ceph-mon[37701]: pgmap v618: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:11:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:53 smithi167 ceph-mon[29496]: pgmap v618: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:11:55.291 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:11:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:11:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:11:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:55 smithi179 ceph-mon[33312]: pgmap v619: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:11:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-71"}]: dispatch 2024-06-18T13:11:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-71"}]: dispatch 2024-06-18T13:11:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:11:55.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:55 smithi167 ceph-mon[37701]: pgmap v619: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:11:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-71"}]: dispatch 2024-06-18T13:11:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-71"}]: dispatch 2024-06-18T13:11:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:11:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:55 smithi167 ceph-mon[29496]: pgmap v619: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:11:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:11:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-71"}]: dispatch 2024-06-18T13:11:55.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-71"}]: dispatch 2024-06-18T13:11:55.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:11:56.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:56 smithi179 ceph-mon[33312]: osdmap e424: 8 total, 8 up, 8 in 2024-06-18T13:11:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:56 smithi167 ceph-mon[37701]: osdmap e424: 8 total, 8 up, 8 in 2024-06-18T13:11:56.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:56 smithi167 ceph-mon[29496]: osdmap e424: 8 total, 8 up, 8 in 2024-06-18T13:11:57.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:57 smithi179 ceph-mon[33312]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-06-18T13:11:57.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:57 smithi179 ceph-mon[33312]: osdmap e425: 8 total, 8 up, 8 in 2024-06-18T13:11:57.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:57.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:57.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:57 smithi167 ceph-mon[37701]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-06-18T13:11:57.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:57 smithi167 ceph-mon[37701]: osdmap e425: 8 total, 8 up, 8 in 2024-06-18T13:11:57.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:57.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:57.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:57 smithi167 ceph-mon[29496]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-06-18T13:11:57.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:57 smithi167 ceph-mon[29496]: osdmap e425: 8 total, 8 up, 8 in 2024-06-18T13:11:57.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:57.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:11:58.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:58.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:58 smithi179 ceph-mon[33312]: osdmap e426: 8 total, 8 up, 8 in 2024-06-18T13:11:58.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:58.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:58.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:58 smithi167 ceph-mon[37701]: osdmap e426: 8 total, 8 up, 8 in 2024-06-18T13:11:58.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:58.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:11:58.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:58 smithi167 ceph-mon[29496]: osdmap e426: 8 total, 8 up, 8 in 2024-06-18T13:11:58.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:11:59.328 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:11:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:11:59.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:59 smithi179 ceph-mon[33312]: pgmap v624: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:59.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:11:59 smithi179 ceph-mon[33312]: osdmap e427: 8 total, 8 up, 8 in 2024-06-18T13:11:59.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:59 smithi167 ceph-mon[37701]: pgmap v624: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:11:59 smithi167 ceph-mon[37701]: osdmap e427: 8 total, 8 up, 8 in 2024-06-18T13:11:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:59 smithi167 ceph-mon[29496]: pgmap v624: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:11:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:11:59 smithi167 ceph-mon[29496]: osdmap e427: 8 total, 8 up, 8 in 2024-06-18T13:12:00.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:00.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:00 smithi179 ceph-mon[33312]: osdmap e428: 8 total, 8 up, 8 in 2024-06-18T13:12:00.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:00 smithi167 ceph-mon[37701]: osdmap e428: 8 total, 8 up, 8 in 2024-06-18T13:12:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:00 smithi167 ceph-mon[29496]: osdmap e428: 8 total, 8 up, 8 in 2024-06-18T13:12:01.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:01 smithi179 ceph-mon[33312]: pgmap v627: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:12:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:01 smithi167 ceph-mon[37701]: pgmap v627: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:12:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:01 smithi167 ceph-mon[29496]: pgmap v627: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:12:03.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:03 smithi179 ceph-mon[33312]: pgmap v628: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 865 B/s rd, 1.0 KiB/s wr, 1 op/s 2024-06-18T13:12:03.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:03 smithi167 ceph-mon[29496]: pgmap v628: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 865 B/s rd, 1.0 KiB/s wr, 1 op/s 2024-06-18T13:12:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:03 smithi167 ceph-mon[37701]: pgmap v628: 260 pgs: 260 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 865 B/s rd, 1.0 KiB/s wr, 1 op/s 2024-06-18T13:12:05.341 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:12:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:12:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:12:05.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:05 smithi179 ceph-mon[33312]: pgmap v629: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-06-18T13:12:05.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:05 smithi167 ceph-mon[37701]: pgmap v629: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-06-18T13:12:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:05 smithi167 ceph-mon[29496]: pgmap v629: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-06-18T13:12:07.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:07 smithi179 ceph-mon[33312]: pgmap v630: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-06-18T13:12:07.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:07 smithi167 ceph-mon[37701]: pgmap v630: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-06-18T13:12:07.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:07 smithi167 ceph-mon[29496]: pgmap v630: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-06-18T13:12:09.347 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:12:08 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:12:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:09 smithi179 ceph-mon[33312]: pgmap v631: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 723 B/s wr, 3 op/s 2024-06-18T13:12:09.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:09 smithi167 ceph-mon[37701]: pgmap v631: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 723 B/s wr, 3 op/s 2024-06-18T13:12:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:09 smithi167 ceph-mon[29496]: pgmap v631: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 723 B/s wr, 3 op/s 2024-06-18T13:12:10.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-73"}]: dispatch 2024-06-18T13:12:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-73"}]: dispatch 2024-06-18T13:12:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:12:10.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-73"}]: dispatch 2024-06-18T13:12:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-73"}]: dispatch 2024-06-18T13:12:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:12:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-73"}]: dispatch 2024-06-18T13:12:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-73"}]: dispatch 2024-06-18T13:12:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:12:11.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:11 smithi179 ceph-mon[33312]: pgmap v632: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 93 B/s wr, 2 op/s 2024-06-18T13:12:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:11 smithi179 ceph-mon[33312]: osdmap e429: 8 total, 8 up, 8 in 2024-06-18T13:12:11.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:11 smithi167 ceph-mon[37701]: pgmap v632: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 93 B/s wr, 2 op/s 2024-06-18T13:12:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:11 smithi167 ceph-mon[37701]: osdmap e429: 8 total, 8 up, 8 in 2024-06-18T13:12:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:11 smithi167 ceph-mon[29496]: pgmap v632: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 93 B/s wr, 2 op/s 2024-06-18T13:12:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:11 smithi167 ceph-mon[29496]: osdmap e429: 8 total, 8 up, 8 in 2024-06-18T13:12:12.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:12 smithi179 ceph-mon[33312]: osdmap e430: 8 total, 8 up, 8 in 2024-06-18T13:12:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:12.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:12 smithi167 ceph-mon[37701]: osdmap e430: 8 total, 8 up, 8 in 2024-06-18T13:12:12.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:12.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:12 smithi167 ceph-mon[29496]: osdmap e430: 8 total, 8 up, 8 in 2024-06-18T13:12:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:13.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:13 smithi179 ceph-mon[33312]: pgmap v635: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:13.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:13 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:13.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:13.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:13 smithi179 ceph-mon[33312]: osdmap e431: 8 total, 8 up, 8 in 2024-06-18T13:12:13.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:13 smithi167 ceph-mon[37701]: pgmap v635: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:13 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:13 smithi167 ceph-mon[37701]: osdmap e431: 8 total, 8 up, 8 in 2024-06-18T13:12:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:13 smithi167 ceph-mon[29496]: pgmap v635: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:13 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:13 smithi167 ceph-mon[29496]: osdmap e431: 8 total, 8 up, 8 in 2024-06-18T13:12:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:14.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:14 smithi167 ceph-mon[37701]: osdmap e432: 8 total, 8 up, 8 in 2024-06-18T13:12:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:14 smithi167 ceph-mon[29496]: osdmap e432: 8 total, 8 up, 8 in 2024-06-18T13:12:14.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:14 smithi179 ceph-mon[33312]: osdmap e432: 8 total, 8 up, 8 in 2024-06-18T13:12:15.403 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:12:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:12:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:12:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:15 smithi167 ceph-mon[37701]: pgmap v638: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:15 smithi167 ceph-mon[37701]: osdmap e433: 8 total, 8 up, 8 in 2024-06-18T13:12:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-75"}]: dispatch 2024-06-18T13:12:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-75"}]: dispatch 2024-06-18T13:12:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:15 smithi167 ceph-mon[29496]: pgmap v638: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:15 smithi167 ceph-mon[29496]: osdmap e433: 8 total, 8 up, 8 in 2024-06-18T13:12:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:15.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-75"}]: dispatch 2024-06-18T13:12:15.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-75"}]: dispatch 2024-06-18T13:12:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:15 smithi179 ceph-mon[33312]: pgmap v638: 260 pgs: 260 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:15 smithi179 ceph-mon[33312]: osdmap e433: 8 total, 8 up, 8 in 2024-06-18T13:12:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-75"}]: dispatch 2024-06-18T13:12:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-75"}]: dispatch 2024-06-18T13:12:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:16 smithi167 ceph-mon[29496]: osdmap e434: 8 total, 8 up, 8 in 2024-06-18T13:12:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:16 smithi167 ceph-mon[37701]: osdmap e434: 8 total, 8 up, 8 in 2024-06-18T13:12:16.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:16 smithi179 ceph-mon[33312]: osdmap e434: 8 total, 8 up, 8 in 2024-06-18T13:12:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:17 smithi167 ceph-mon[37701]: pgmap v641: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:12:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:17 smithi167 ceph-mon[37701]: osdmap e435: 8 total, 8 up, 8 in 2024-06-18T13:12:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:17.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:17 smithi167 ceph-mon[29496]: pgmap v641: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:12:17.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:17 smithi167 ceph-mon[29496]: osdmap e435: 8 total, 8 up, 8 in 2024-06-18T13:12:17.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:17.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:17 smithi179 ceph-mon[33312]: pgmap v641: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:12:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:17 smithi179 ceph-mon[33312]: osdmap e435: 8 total, 8 up, 8 in 2024-06-18T13:12:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:18 smithi167 ceph-mon[37701]: osdmap e436: 8 total, 8 up, 8 in 2024-06-18T13:12:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:18 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:18 smithi167 ceph-mon[37701]: osdmap e437: 8 total, 8 up, 8 in 2024-06-18T13:12:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:18 smithi167 ceph-mon[29496]: osdmap e436: 8 total, 8 up, 8 in 2024-06-18T13:12:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:18 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:18 smithi167 ceph-mon[29496]: osdmap e437: 8 total, 8 up, 8 in 2024-06-18T13:12:18.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:18 smithi179 ceph-mon[33312]: osdmap e436: 8 total, 8 up, 8 in 2024-06-18T13:12:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:18 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:18 smithi179 ceph-mon[33312]: osdmap e437: 8 total, 8 up, 8 in 2024-06-18T13:12:19.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:12:18 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:12:19.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:19 smithi167 ceph-mon[29496]: pgmap v644: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-77"}]: dispatch 2024-06-18T13:12:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-77"}]: dispatch 2024-06-18T13:12:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:19 smithi167 ceph-mon[37701]: pgmap v644: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-77"}]: dispatch 2024-06-18T13:12:19.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-77"}]: dispatch 2024-06-18T13:12:19.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:19 smithi179 ceph-mon[33312]: pgmap v644: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-77"}]: dispatch 2024-06-18T13:12:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-77"}]: dispatch 2024-06-18T13:12:20.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:20 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:20 smithi167 ceph-mon[37701]: osdmap e438: 8 total, 8 up, 8 in 2024-06-18T13:12:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:20 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:20 smithi167 ceph-mon[29496]: osdmap e438: 8 total, 8 up, 8 in 2024-06-18T13:12:20.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:20 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:20.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:20 smithi179 ceph-mon[33312]: osdmap e438: 8 total, 8 up, 8 in 2024-06-18T13:12:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:21 smithi167 ceph-mon[37701]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-18T13:12:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:21 smithi167 ceph-mon[37701]: osdmap e439: 8 total, 8 up, 8 in 2024-06-18T13:12:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:21.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:21 smithi167 ceph-mon[29496]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-18T13:12:21.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:21 smithi167 ceph-mon[29496]: osdmap e439: 8 total, 8 up, 8 in 2024-06-18T13:12:21.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:21.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:21.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:21 smithi179 ceph-mon[33312]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-18T13:12:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:21 smithi179 ceph-mon[33312]: osdmap e439: 8 total, 8 up, 8 in 2024-06-18T13:12:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:22.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:22 smithi167 ceph-mon[37701]: osdmap e440: 8 total, 8 up, 8 in 2024-06-18T13:12:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:22 smithi167 ceph-mon[37701]: pgmap v650: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-18T13:12:22.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:22.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:22 smithi167 ceph-mon[29496]: osdmap e440: 8 total, 8 up, 8 in 2024-06-18T13:12:22.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:22.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:22 smithi167 ceph-mon[29496]: pgmap v650: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-18T13:12:22.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:22.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:22 smithi179 ceph-mon[33312]: osdmap e440: 8 total, 8 up, 8 in 2024-06-18T13:12:22.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:22 smithi179 ceph-mon[33312]: pgmap v650: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-18T13:12:23.787 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:23 smithi179 ceph-mon[33312]: osdmap e441: 8 total, 8 up, 8 in 2024-06-18T13:12:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:23 smithi167 ceph-mon[37701]: osdmap e441: 8 total, 8 up, 8 in 2024-06-18T13:12:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:23 smithi167 ceph-mon[29496]: osdmap e441: 8 total, 8 up, 8 in 2024-06-18T13:12:24.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:24 smithi179 ceph-mon[33312]: osdmap e442: 8 total, 8 up, 8 in 2024-06-18T13:12:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.1f"}]: dispatch 2024-06-18T13:12:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:24 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.1f"}]: dispatch 2024-06-18T13:12:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:24 smithi179 ceph-mon[33312]: 304.1f deep-scrub starts 2024-06-18T13:12:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:24 smithi179 ceph-mon[33312]: 304.1f deep-scrub ok 2024-06-18T13:12:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:24 smithi179 ceph-mon[33312]: pgmap v653: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-18T13:12:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:24 smithi167 ceph-mon[37701]: osdmap e442: 8 total, 8 up, 8 in 2024-06-18T13:12:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.1f"}]: dispatch 2024-06-18T13:12:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:24 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.1f"}]: dispatch 2024-06-18T13:12:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:24 smithi167 ceph-mon[37701]: 304.1f deep-scrub starts 2024-06-18T13:12:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:24 smithi167 ceph-mon[37701]: 304.1f deep-scrub ok 2024-06-18T13:12:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:24 smithi167 ceph-mon[37701]: pgmap v653: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-18T13:12:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:24 smithi167 ceph-mon[29496]: osdmap e442: 8 total, 8 up, 8 in 2024-06-18T13:12:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.1f"}]: dispatch 2024-06-18T13:12:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:24 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.1f"}]: dispatch 2024-06-18T13:12:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:24 smithi167 ceph-mon[29496]: 304.1f deep-scrub starts 2024-06-18T13:12:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:24 smithi167 ceph-mon[29496]: 304.1f deep-scrub ok 2024-06-18T13:12:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:24 smithi167 ceph-mon[29496]: pgmap v653: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-18T13:12:25.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:12:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:12:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:12:25.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:12:25.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:12:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:12:26.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:26 smithi179 ceph-mon[33312]: pgmap v654: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 885 B/s rd, 885 B/s wr, 2 op/s 2024-06-18T13:12:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:26 smithi167 ceph-mon[29496]: pgmap v654: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 885 B/s rd, 885 B/s wr, 2 op/s 2024-06-18T13:12:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:26 smithi167 ceph-mon[37701]: pgmap v654: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 885 B/s rd, 885 B/s wr, 2 op/s 2024-06-18T13:12:29.286 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:12:28 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:12:29.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:29 smithi179 ceph-mon[33312]: pgmap v655: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 755 B/s rd, 755 B/s wr, 2 op/s 2024-06-18T13:12:29.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:29 smithi167 ceph-mon[37701]: pgmap v655: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 755 B/s rd, 755 B/s wr, 2 op/s 2024-06-18T13:12:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:29 smithi167 ceph-mon[29496]: pgmap v655: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 755 B/s rd, 755 B/s wr, 2 op/s 2024-06-18T13:12:30.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:31.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:31 smithi179 ceph-mon[33312]: pgmap v656: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-06-18T13:12:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:31 smithi167 ceph-mon[37701]: pgmap v656: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-06-18T13:12:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:31 smithi167 ceph-mon[29496]: pgmap v656: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-06-18T13:12:33.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:33 smithi179 ceph-mon[33312]: pgmap v657: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 524 B/s wr, 2 op/s 2024-06-18T13:12:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:33 smithi167 ceph-mon[37701]: pgmap v657: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 524 B/s wr, 2 op/s 2024-06-18T13:12:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:33 smithi167 ceph-mon[29496]: pgmap v657: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 524 B/s wr, 2 op/s 2024-06-18T13:12:35.300 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:12:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:12:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:12:35.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:35 smithi179 ceph-mon[33312]: pgmap v658: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 475 B/s wr, 2 op/s 2024-06-18T13:12:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:35 smithi167 ceph-mon[29496]: pgmap v658: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 475 B/s wr, 2 op/s 2024-06-18T13:12:35.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:35 smithi167 ceph-mon[37701]: pgmap v658: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 475 B/s wr, 2 op/s 2024-06-18T13:12:37.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:37 smithi179 ceph-mon[33312]: pgmap v659: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:12:37.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:37 smithi167 ceph-mon[37701]: pgmap v659: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:12:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:37 smithi167 ceph-mon[29496]: pgmap v659: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:12:39.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:12:38 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:12:39.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:39 smithi179 ceph-mon[33312]: pgmap v660: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:12:39.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:39 smithi167 ceph-mon[37701]: pgmap v660: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:12:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:39 smithi167 ceph-mon[29496]: pgmap v660: 260 pgs: 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:12:40.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:12:40.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:40.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:12:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:12:41.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:41 smithi179 ceph-mon[33312]: pgmap v661: 260 pgs: 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:12:41.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:41 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:12:41.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:41 smithi167 ceph-mon[29496]: pgmap v661: 260 pgs: 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:12:41.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:41 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:12:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:41 smithi167 ceph-mon[37701]: pgmap v661: 260 pgs: 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:12:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:41 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:12:42.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:12:42.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:12:42.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:12:42.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:12:42.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:12:42.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:12:42.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:12:42.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:12:42.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:12:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:43 smithi167 ceph-mon[37701]: pgmap v662: 260 pgs: 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:12:43.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:43 smithi167 ceph-mon[29496]: pgmap v662: 260 pgs: 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:12:43.727 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:12:43 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3[58367]: 2024-06-18T13:12:43.509+0000 7f8916e16640 -1 snap_mapper.add_oid found existing snaps mapped on 304:fabf6cb2:test-rados-api-smithi167-74266-80::foo:2, removing 2024-06-18T13:12:43.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:43 smithi179 ceph-mon[33312]: pgmap v662: 260 pgs: 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:12:43.887 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:12:43 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:12:43.509+0000 7f5fefe44640 -1 snap_mapper.add_oid found existing snaps mapped on 304:fabf6cb2:test-rados-api-smithi167-74266-80::foo:2, removing 2024-06-18T13:12:43.887 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:12:43 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:12:43.509+0000 7f2718220640 -1 snap_mapper.add_oid found existing snaps mapped on 304:fabf6cb2:test-rados-api-smithi167-74266-80::foo:2, removing 2024-06-18T13:12:44.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:44.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:44 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:44.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-79"}]: dispatch 2024-06-18T13:12:44.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:44 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-79"}]: dispatch 2024-06-18T13:12:44.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:44.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:44 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:44.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-79"}]: dispatch 2024-06-18T13:12:44.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:44 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-79"}]: dispatch 2024-06-18T13:12:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:44 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-79"}]: dispatch 2024-06-18T13:12:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:44 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-79"}]: dispatch 2024-06-18T13:12:45.430 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:12:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:12:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:12:45.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:45 smithi167 ceph-mon[37701]: pgmap v663: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 85 B/s wr, 61 op/s 2024-06-18T13:12:45.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:45 smithi167 ceph-mon[37701]: osdmap e443: 8 total, 8 up, 8 in 2024-06-18T13:12:45.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:45 smithi167 ceph-mon[29496]: pgmap v663: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 85 B/s wr, 61 op/s 2024-06-18T13:12:45.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:45 smithi167 ceph-mon[29496]: osdmap e443: 8 total, 8 up, 8 in 2024-06-18T13:12:45.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:45 smithi179 ceph-mon[33312]: pgmap v663: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 85 B/s wr, 61 op/s 2024-06-18T13:12:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:45 smithi179 ceph-mon[33312]: osdmap e443: 8 total, 8 up, 8 in 2024-06-18T13:12:46.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:46 smithi167 ceph-mon[37701]: osdmap e444: 8 total, 8 up, 8 in 2024-06-18T13:12:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:46 smithi167 ceph-mon[29496]: osdmap e444: 8 total, 8 up, 8 in 2024-06-18T13:12:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:46.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:46 smithi179 ceph-mon[33312]: osdmap e444: 8 total, 8 up, 8 in 2024-06-18T13:12:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:47 smithi167 ceph-mon[37701]: pgmap v666: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s 2024-06-18T13:12:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:47 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:47 smithi167 ceph-mon[37701]: osdmap e445: 8 total, 8 up, 8 in 2024-06-18T13:12:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:12:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:12:47.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:47 smithi167 ceph-mon[29496]: pgmap v666: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s 2024-06-18T13:12:47.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:47 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:47.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:47.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:47 smithi167 ceph-mon[29496]: osdmap e445: 8 total, 8 up, 8 in 2024-06-18T13:12:47.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:47.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:12:47.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:12:47.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:47 smithi179 ceph-mon[33312]: pgmap v666: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s 2024-06-18T13:12:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:47 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:47 smithi179 ceph-mon[33312]: osdmap e445: 8 total, 8 up, 8 in 2024-06-18T13:12:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:12:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:12:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:12:48.779 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[37701]: osdmap e446: 8 total, 8 up, 8 in 2024-06-18T13:12:48.779 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:48.779 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:48.779 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:12:48.779 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[37701]: osdmap e447: 8 total, 8 up, 8 in 2024-06-18T13:12:48.780 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:12:48.780 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:12:48.780 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:12:48.780 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[29496]: osdmap e446: 8 total, 8 up, 8 in 2024-06-18T13:12:48.780 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:48.780 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:48.780 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:12:48.780 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[29496]: osdmap e447: 8 total, 8 up, 8 in 2024-06-18T13:12:48.780 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:12:48.781 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:12:48.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:12:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:48 smithi179 ceph-mon[33312]: osdmap e446: 8 total, 8 up, 8 in 2024-06-18T13:12:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:12:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:48 smithi179 ceph-mon[33312]: osdmap e447: 8 total, 8 up, 8 in 2024-06-18T13:12:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:12:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:12:49.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:12:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:12:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:49 smithi167 ceph-mon[37701]: pgmap v669: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:49 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:12:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:49 smithi167 ceph-mon[37701]: osdmap e448: 8 total, 8 up, 8 in 2024-06-18T13:12:49.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:12:49.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:12:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:49 smithi167 ceph-mon[29496]: pgmap v669: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:49 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:12:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:49 smithi167 ceph-mon[29496]: osdmap e448: 8 total, 8 up, 8 in 2024-06-18T13:12:49.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:12:49.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:12:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:49 smithi179 ceph-mon[33312]: pgmap v669: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:49 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:12:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:49 smithi179 ceph-mon[33312]: osdmap e448: 8 total, 8 up, 8 in 2024-06-18T13:12:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:12:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:12:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:51 smithi167 ceph-mon[37701]: pgmap v672: 260 pgs: 260 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:12:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:51 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:12:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:51 smithi167 ceph-mon[37701]: osdmap e449: 8 total, 8 up, 8 in 2024-06-18T13:12:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:51 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-06-18T13:12:51.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:51 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-06-18T13:12:51.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:51 smithi167 ceph-mon[29496]: pgmap v672: 260 pgs: 260 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:12:51.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:51 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:12:51.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:51 smithi167 ceph-mon[29496]: osdmap e449: 8 total, 8 up, 8 in 2024-06-18T13:12:51.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:51 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-06-18T13:12:51.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:51 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-06-18T13:12:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:51 smithi179 ceph-mon[33312]: pgmap v672: 260 pgs: 260 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:12:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:51 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:12:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:51 smithi179 ceph-mon[33312]: osdmap e449: 8 total, 8 up, 8 in 2024-06-18T13:12:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:51 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-06-18T13:12:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:51 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-06-18T13:12:52.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:52 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-06-18T13:12:52.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:52 smithi167 ceph-mon[29496]: osdmap e450: 8 total, 8 up, 8 in 2024-06-18T13:12:52.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:52 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-06-18T13:12:52.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:52 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-06-18T13:12:52.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:52 smithi167 ceph-mon[29496]: pgmap v675: 260 pgs: 260 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:12:52.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:52 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-06-18T13:12:52.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:52 smithi167 ceph-mon[37701]: osdmap e450: 8 total, 8 up, 8 in 2024-06-18T13:12:52.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:52 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-06-18T13:12:52.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:52 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-06-18T13:12:52.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:52 smithi167 ceph-mon[37701]: pgmap v675: 260 pgs: 260 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:12:52.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:52 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-06-18T13:12:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:52 smithi179 ceph-mon[33312]: osdmap e450: 8 total, 8 up, 8 in 2024-06-18T13:12:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:52 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-06-18T13:12:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:52 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-06-18T13:12:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:52 smithi179 ceph-mon[33312]: pgmap v675: 260 pgs: 260 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:12:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-06-18T13:12:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:53 smithi179 ceph-mon[33312]: osdmap e451: 8 total, 8 up, 8 in 2024-06-18T13:12:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:12:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:12:53.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:53 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-06-18T13:12:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:53 smithi167 ceph-mon[37701]: osdmap e451: 8 total, 8 up, 8 in 2024-06-18T13:12:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:12:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:12:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:53 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:53.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-06-18T13:12:53.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:53 smithi167 ceph-mon[29496]: osdmap e451: 8 total, 8 up, 8 in 2024-06-18T13:12:53.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:12:53.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:12:53.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:53 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:12:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:54 smithi179 ceph-mon[33312]: osdmap e452: 8 total, 8 up, 8 in 2024-06-18T13:12:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-81"}]: dispatch 2024-06-18T13:12:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-81"}]: dispatch 2024-06-18T13:12:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:54 smithi179 ceph-mon[33312]: pgmap v678: 260 pgs: 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 9.7 KiB/s rd, 15 KiB/s wr, 40 op/s 2024-06-18T13:12:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:12:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:54 smithi167 ceph-mon[29496]: osdmap e452: 8 total, 8 up, 8 in 2024-06-18T13:12:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-81"}]: dispatch 2024-06-18T13:12:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-81"}]: dispatch 2024-06-18T13:12:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:54 smithi167 ceph-mon[29496]: pgmap v678: 260 pgs: 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 9.7 KiB/s rd, 15 KiB/s wr, 40 op/s 2024-06-18T13:12:54.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:12:54.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:54 smithi167 ceph-mon[37701]: osdmap e452: 8 total, 8 up, 8 in 2024-06-18T13:12:54.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:54.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:54.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-81"}]: dispatch 2024-06-18T13:12:54.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-81"}]: dispatch 2024-06-18T13:12:54.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:54 smithi167 ceph-mon[37701]: pgmap v678: 260 pgs: 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 9.7 KiB/s rd, 15 KiB/s wr, 40 op/s 2024-06-18T13:12:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:12:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:12:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:12:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:55 smithi179 ceph-mon[33312]: osdmap e453: 8 total, 8 up, 8 in 2024-06-18T13:12:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:12:55.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:55 smithi167 ceph-mon[29496]: osdmap e453: 8 total, 8 up, 8 in 2024-06-18T13:12:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:12:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:55 smithi167 ceph-mon[37701]: osdmap e453: 8 total, 8 up, 8 in 2024-06-18T13:12:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:12:56.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:56 smithi179 ceph-mon[33312]: osdmap e454: 8 total, 8 up, 8 in 2024-06-18T13:12:56.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:56 smithi179 ceph-mon[33312]: pgmap v681: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 9.7 KiB/s rd, 12 KiB/s wr, 38 op/s 2024-06-18T13:12:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:56 smithi167 ceph-mon[37701]: osdmap e454: 8 total, 8 up, 8 in 2024-06-18T13:12:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:56 smithi167 ceph-mon[37701]: pgmap v681: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 9.7 KiB/s rd, 12 KiB/s wr, 38 op/s 2024-06-18T13:12:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:56 smithi167 ceph-mon[29496]: osdmap e454: 8 total, 8 up, 8 in 2024-06-18T13:12:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:12:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:56 smithi167 ceph-mon[29496]: pgmap v681: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 9.7 KiB/s rd, 12 KiB/s wr, 38 op/s 2024-06-18T13:12:57.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:57.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:57 smithi179 ceph-mon[33312]: osdmap e455: 8 total, 8 up, 8 in 2024-06-18T13:12:57.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:57.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:12:57.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:12:57.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:57 smithi167 ceph-mon[37701]: osdmap e455: 8 total, 8 up, 8 in 2024-06-18T13:12:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:12:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:12:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:12:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:57 smithi167 ceph-mon[29496]: osdmap e455: 8 total, 8 up, 8 in 2024-06-18T13:12:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:12:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:12:57.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:12:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:12:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:58 smithi179 ceph-mon[33312]: osdmap e456: 8 total, 8 up, 8 in 2024-06-18T13:12:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:58 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:58.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:58 smithi179 ceph-mon[33312]: pgmap v684: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:12:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:58 smithi167 ceph-mon[37701]: osdmap e456: 8 total, 8 up, 8 in 2024-06-18T13:12:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:58 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:58 smithi167 ceph-mon[37701]: pgmap v684: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:12:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:58 smithi167 ceph-mon[29496]: osdmap e456: 8 total, 8 up, 8 in 2024-06-18T13:12:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:12:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:58 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:12:58.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:58 smithi167 ceph-mon[29496]: pgmap v684: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:12:59.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:12:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:12:59.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:12:59.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:59 smithi179 ceph-mon[33312]: osdmap e457: 8 total, 8 up, 8 in 2024-06-18T13:12:59.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:59 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:12:59.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:12:59.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:12:59 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:12:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:59 smithi167 ceph-mon[37701]: osdmap e457: 8 total, 8 up, 8 in 2024-06-18T13:12:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:59 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:12:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:12:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:12:59 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:12:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:12:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:59 smithi167 ceph-mon[29496]: osdmap e457: 8 total, 8 up, 8 in 2024-06-18T13:12:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:59 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:12:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:12:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:12:59 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:00.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:13:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:00 smithi179 ceph-mon[33312]: osdmap e458: 8 total, 8 up, 8 in 2024-06-18T13:13:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:00 smithi179 ceph-mon[33312]: pgmap v687: 260 pgs: 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:13:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:00 smithi167 ceph-mon[37701]: osdmap e458: 8 total, 8 up, 8 in 2024-06-18T13:13:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:00 smithi167 ceph-mon[37701]: pgmap v687: 260 pgs: 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:13:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:00 smithi167 ceph-mon[29496]: osdmap e458: 8 total, 8 up, 8 in 2024-06-18T13:13:00.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:00.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:00.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:00 smithi167 ceph-mon[29496]: pgmap v687: 260 pgs: 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:01.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:13:01.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:01 smithi179 ceph-mon[33312]: osdmap e459: 8 total, 8 up, 8 in 2024-06-18T13:13:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:13:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:01 smithi167 ceph-mon[37701]: osdmap e459: 8 total, 8 up, 8 in 2024-06-18T13:13:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:13:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:01 smithi167 ceph-mon[29496]: osdmap e459: 8 total, 8 up, 8 in 2024-06-18T13:13:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:02 smithi179 ceph-mon[33312]: osdmap e460: 8 total, 8 up, 8 in 2024-06-18T13:13:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:02 smithi179 ceph-mon[33312]: pgmap v690: 260 pgs: 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:02 smithi167 ceph-mon[37701]: osdmap e460: 8 total, 8 up, 8 in 2024-06-18T13:13:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:02 smithi167 ceph-mon[37701]: pgmap v690: 260 pgs: 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:02 smithi167 ceph-mon[29496]: osdmap e460: 8 total, 8 up, 8 in 2024-06-18T13:13:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:02 smithi167 ceph-mon[29496]: pgmap v690: 260 pgs: 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:03.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:03 smithi179 ceph-mon[33312]: osdmap e461: 8 total, 8 up, 8 in 2024-06-18T13:13:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:03 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:03 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:03 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-83"}]: dispatch 2024-06-18T13:13:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:03 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-83"}]: dispatch 2024-06-18T13:13:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:03 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:03 smithi167 ceph-mon[37701]: osdmap e461: 8 total, 8 up, 8 in 2024-06-18T13:13:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:03 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:03 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:03 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-83"}]: dispatch 2024-06-18T13:13:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:03 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-83"}]: dispatch 2024-06-18T13:13:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:03 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:03 smithi167 ceph-mon[29496]: osdmap e461: 8 total, 8 up, 8 in 2024-06-18T13:13:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:03 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:03 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:03.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:03 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-83"}]: dispatch 2024-06-18T13:13:03.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:03 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-83"}]: dispatch 2024-06-18T13:13:03.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:03 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:04 smithi179 ceph-mon[33312]: osdmap e462: 8 total, 8 up, 8 in 2024-06-18T13:13:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:04 smithi179 ceph-mon[33312]: pgmap v693: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-06-18T13:13:04.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:04 smithi167 ceph-mon[37701]: osdmap e462: 8 total, 8 up, 8 in 2024-06-18T13:13:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:04 smithi167 ceph-mon[37701]: pgmap v693: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-06-18T13:13:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:04 smithi167 ceph-mon[29496]: osdmap e462: 8 total, 8 up, 8 in 2024-06-18T13:13:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:04 smithi167 ceph-mon[29496]: pgmap v693: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-06-18T13:13:05.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:13:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:13:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:13:05.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:05 smithi179 ceph-mon[33312]: osdmap e463: 8 total, 8 up, 8 in 2024-06-18T13:13:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:05 smithi167 ceph-mon[37701]: osdmap e463: 8 total, 8 up, 8 in 2024-06-18T13:13:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:05 smithi167 ceph-mon[29496]: osdmap e463: 8 total, 8 up, 8 in 2024-06-18T13:13:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:06.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:06 smithi179 ceph-mon[33312]: osdmap e464: 8 total, 8 up, 8 in 2024-06-18T13:13:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:06 smithi179 ceph-mon[33312]: pgmap v696: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-06-18T13:13:06.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:06 smithi167 ceph-mon[37701]: osdmap e464: 8 total, 8 up, 8 in 2024-06-18T13:13:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:06 smithi167 ceph-mon[37701]: pgmap v696: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-06-18T13:13:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:06 smithi167 ceph-mon[29496]: osdmap e464: 8 total, 8 up, 8 in 2024-06-18T13:13:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:06.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:06.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:06 smithi167 ceph-mon[29496]: pgmap v696: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-06-18T13:13:07.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:13:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:07 smithi179 ceph-mon[33312]: osdmap e465: 8 total, 8 up, 8 in 2024-06-18T13:13:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:07.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:13:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:07 smithi167 ceph-mon[37701]: osdmap e465: 8 total, 8 up, 8 in 2024-06-18T13:13:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:13:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:07 smithi167 ceph-mon[29496]: osdmap e465: 8 total, 8 up, 8 in 2024-06-18T13:13:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:08.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:13:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:13:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:08 smithi179 ceph-mon[33312]: osdmap e466: 8 total, 8 up, 8 in 2024-06-18T13:13:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:13:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:08 smithi179 ceph-mon[33312]: pgmap v699: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:13:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:13:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:08 smithi179 ceph-mon[33312]: osdmap e467: 8 total, 8 up, 8 in 2024-06-18T13:13:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:13:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:13:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[37701]: osdmap e466: 8 total, 8 up, 8 in 2024-06-18T13:13:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:13:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[37701]: pgmap v699: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:13:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:13:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[37701]: osdmap e467: 8 total, 8 up, 8 in 2024-06-18T13:13:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_tier","val": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:13:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:13:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[29496]: osdmap e466: 8 total, 8 up, 8 in 2024-06-18T13:13:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:13:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[29496]: pgmap v699: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:13:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:13:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[29496]: osdmap e467: 8 total, 8 up, 8 in 2024-06-18T13:13:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:09.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:13:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:13:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:09 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:13:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:09 smithi167 ceph-mon[37701]: osdmap e468: 8 total, 8 up, 8 in 2024-06-18T13:13:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:09 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:13:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:09 smithi167 ceph-mon[29496]: osdmap e468: 8 total, 8 up, 8 in 2024-06-18T13:13:10.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:09 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:10.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:13:10.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:09 smithi179 ceph-mon[33312]: osdmap e468: 8 total, 8 up, 8 in 2024-06-18T13:13:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:13:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:10 smithi167 ceph-mon[37701]: pgmap v702: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:13:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:10 smithi167 ceph-mon[37701]: osdmap e469: 8 total, 8 up, 8 in 2024-06-18T13:13:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:13:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:10 smithi167 ceph-mon[29496]: pgmap v702: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:13:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:10 smithi167 ceph-mon[29496]: osdmap e469: 8 total, 8 up, 8 in 2024-06-18T13:13:11.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:13:11.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:10 smithi179 ceph-mon[33312]: pgmap v702: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:13:11.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:10 smithi179 ceph-mon[33312]: osdmap e469: 8 total, 8 up, 8 in 2024-06-18T13:13:12.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:12 smithi179 ceph-mon[33312]: osdmap e470: 8 total, 8 up, 8 in 2024-06-18T13:13:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-85"}]: dispatch 2024-06-18T13:13:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-85"}]: dispatch 2024-06-18T13:13:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:12 smithi179 ceph-mon[33312]: pgmap v705: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:13:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:12 smithi167 ceph-mon[37701]: osdmap e470: 8 total, 8 up, 8 in 2024-06-18T13:13:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-85"}]: dispatch 2024-06-18T13:13:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-85"}]: dispatch 2024-06-18T13:13:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:12 smithi167 ceph-mon[37701]: pgmap v705: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:13:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:12 smithi167 ceph-mon[29496]: osdmap e470: 8 total, 8 up, 8 in 2024-06-18T13:13:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:12.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-85"}]: dispatch 2024-06-18T13:13:12.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-85"}]: dispatch 2024-06-18T13:13:12.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:12 smithi167 ceph-mon[29496]: pgmap v705: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:13:13.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:13 smithi179 ceph-mon[33312]: osdmap e471: 8 total, 8 up, 8 in 2024-06-18T13:13:13.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:13 smithi167 ceph-mon[29496]: osdmap e471: 8 total, 8 up, 8 in 2024-06-18T13:13:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:13 smithi167 ceph-mon[37701]: osdmap e471: 8 total, 8 up, 8 in 2024-06-18T13:13:14.631 INFO:tasks.workunit.client.0.smithi167.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18307 ms) 2024-06-18T13:13:14.631 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-06-18T13:13:14.631 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22181 ms) 2024-06-18T13:13:14.632 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-06-18T13:13:14.632 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3035 ms) 2024-06-18T13:13:14.632 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-06-18T13:13:14.632 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.632 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3009 ms) 2024-06-18T13:13:14.632 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-06-18T13:13:14.632 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3016 ms) 2024-06-18T13:13:14.632 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-06-18T13:13:14.632 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.632 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3024 ms) 2024-06-18T13:13:14.632 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-06-18T13:13:14.632 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.633 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4031 ms) 2024-06-18T13:13:14.633 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-06-18T13:13:14.633 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.633 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (38936 ms) 2024-06-18T13:13:14.633 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-06-18T13:13:14.633 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.633 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (17076 ms) 2024-06-18T13:13:14.633 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-06-18T13:13:14.633 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.633 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4019 ms) 2024-06-18T13:13:14.633 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-06-18T13:13:14.633 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.633 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3034 ms) 2024-06-18T13:13:14.634 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-06-18T13:13:14.634 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.634 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24869 ms) 2024-06-18T13:13:14.634 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-06-18T13:13:14.634 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.634 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15056 ms) 2024-06-18T13:13:14.634 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-06-18T13:13:14.634 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.634 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5037 ms) 2024-06-18T13:13:14.634 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-06-18T13:13:14.634 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.634 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4036 ms) 2024-06-18T13:13:14.634 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-06-18T13:13:14.635 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.635 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: waiting for scrubs... 2024-06-18T13:13:14.635 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: done waiting 2024-06-18T13:13:14.635 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25006 ms) 2024-06-18T13:13:14.635 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-06-18T13:13:14.635 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.635 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10049 ms) 2024-06-18T13:13:14.635 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-06-18T13:13:14.635 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.635 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9072 ms) 2024-06-18T13:13:14.635 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-06-18T13:13:14.635 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:13:14.637 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9043 ms) 2024-06-18T13:13:14.637 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-06-18T13:13:14.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:14 smithi179 ceph-mon[33312]: osdmap e472: 8 total, 8 up, 8 in 2024-06-18T13:13:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:14 smithi179 ceph-mon[33312]: pgmap v708: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-18T13:13:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:14.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:14 smithi167 ceph-mon[37701]: osdmap e472: 8 total, 8 up, 8 in 2024-06-18T13:13:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:14 smithi167 ceph-mon[37701]: pgmap v708: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-18T13:13:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:14 smithi167 ceph-mon[29496]: osdmap e472: 8 total, 8 up, 8 in 2024-06-18T13:13:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:14 smithi167 ceph-mon[29496]: pgmap v708: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-18T13:13:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:15.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:13:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:13:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:13:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:15 smithi179 ceph-mon[33312]: osdmap e473: 8 total, 8 up, 8 in 2024-06-18T13:13:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:15 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:15 smithi179 ceph-mon[33312]: osdmap e474: 8 total, 8 up, 8 in 2024-06-18T13:13:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:15 smithi167 ceph-mon[29496]: osdmap e473: 8 total, 8 up, 8 in 2024-06-18T13:13:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:15 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:15 smithi167 ceph-mon[29496]: osdmap e474: 8 total, 8 up, 8 in 2024-06-18T13:13:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:15 smithi167 ceph-mon[37701]: osdmap e473: 8 total, 8 up, 8 in 2024-06-18T13:13:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:15 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:15.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:15.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:15 smithi167 ceph-mon[37701]: osdmap e474: 8 total, 8 up, 8 in 2024-06-18T13:13:15.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:15.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:16 smithi167 ceph-mon[37701]: pgmap v711: 292 pgs: 64 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-18T13:13:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:13:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:16 smithi167 ceph-mon[37701]: osdmap e475: 8 total, 8 up, 8 in 2024-06-18T13:13:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_tier","val": "test-rados-api-smithi167-74266-89-test-flush"}]: dispatch 2024-06-18T13:13:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_tier","val": "test-rados-api-smithi167-74266-89-test-flush"}]: dispatch 2024-06-18T13:13:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:16 smithi167 ceph-mon[29496]: pgmap v711: 292 pgs: 64 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-18T13:13:16.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:13:16.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:16 smithi167 ceph-mon[29496]: osdmap e475: 8 total, 8 up, 8 in 2024-06-18T13:13:16.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_tier","val": "test-rados-api-smithi167-74266-89-test-flush"}]: dispatch 2024-06-18T13:13:16.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_tier","val": "test-rados-api-smithi167-74266-89-test-flush"}]: dispatch 2024-06-18T13:13:17.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:16 smithi179 ceph-mon[33312]: pgmap v711: 292 pgs: 64 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-18T13:13:17.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:13:17.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:16 smithi179 ceph-mon[33312]: osdmap e475: 8 total, 8 up, 8 in 2024-06-18T13:13:17.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_tier","val": "test-rados-api-smithi167-74266-89-test-flush"}]: dispatch 2024-06-18T13:13:17.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_tier","val": "test-rados-api-smithi167-74266-89-test-flush"}]: dispatch 2024-06-18T13:13:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_tier","val": "test-rados-api-smithi167-74266-89-test-flush"}]': finished 2024-06-18T13:13:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:18 smithi179 ceph-mon[33312]: osdmap e476: 8 total, 8 up, 8 in 2024-06-18T13:13:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:13:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:13:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:18 smithi179 ceph-mon[33312]: pgmap v714: 292 pgs: 64 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:13:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_tier","val": "test-rados-api-smithi167-74266-89-test-flush"}]': finished 2024-06-18T13:13:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:18 smithi167 ceph-mon[37701]: osdmap e476: 8 total, 8 up, 8 in 2024-06-18T13:13:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:13:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:13:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:18 smithi167 ceph-mon[37701]: pgmap v714: 292 pgs: 64 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:13:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_tier","val": "test-rados-api-smithi167-74266-89-test-flush"}]': finished 2024-06-18T13:13:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:18 smithi167 ceph-mon[29496]: osdmap e476: 8 total, 8 up, 8 in 2024-06-18T13:13:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:13:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:13:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:18 smithi167 ceph-mon[29496]: pgmap v714: 292 pgs: 64 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:13:19.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:13:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:13:19.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:13:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:19 smithi179 ceph-mon[33312]: osdmap e477: 8 total, 8 up, 8 in 2024-06-18T13:13:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:19 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:13:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:19 smithi167 ceph-mon[37701]: osdmap e477: 8 total, 8 up, 8 in 2024-06-18T13:13:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:19 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:13:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:19 smithi167 ceph-mon[29496]: osdmap e477: 8 total, 8 up, 8 in 2024-06-18T13:13:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:13:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:19 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:13:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:20 smithi179 ceph-mon[33312]: osdmap e478: 8 total, 8 up, 8 in 2024-06-18T13:13:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:20 smithi179 ceph-mon[33312]: pgmap v717: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:13:20.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:13:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:20 smithi167 ceph-mon[29496]: osdmap e478: 8 total, 8 up, 8 in 2024-06-18T13:13:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:20 smithi167 ceph-mon[29496]: pgmap v717: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:13:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:13:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:20 smithi167 ceph-mon[37701]: osdmap e478: 8 total, 8 up, 8 in 2024-06-18T13:13:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:20 smithi167 ceph-mon[37701]: pgmap v717: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:13:21.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:21 smithi179 ceph-mon[33312]: osdmap e479: 8 total, 8 up, 8 in 2024-06-18T13:13:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-87"}]: dispatch 2024-06-18T13:13:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-87"}]: dispatch 2024-06-18T13:13:21.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:21 smithi167 ceph-mon[37701]: osdmap e479: 8 total, 8 up, 8 in 2024-06-18T13:13:21.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-87"}]: dispatch 2024-06-18T13:13:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-87"}]: dispatch 2024-06-18T13:13:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:21 smithi167 ceph-mon[29496]: osdmap e479: 8 total, 8 up, 8 in 2024-06-18T13:13:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-87"}]: dispatch 2024-06-18T13:13:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-87"}]: dispatch 2024-06-18T13:13:22.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:22 smithi167 ceph-mon[37701]: osdmap e480: 8 total, 8 up, 8 in 2024-06-18T13:13:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:22 smithi167 ceph-mon[37701]: pgmap v720: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:13:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:22 smithi167 ceph-mon[29496]: osdmap e480: 8 total, 8 up, 8 in 2024-06-18T13:13:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:22 smithi167 ceph-mon[29496]: pgmap v720: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:13:23.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:22 smithi179 ceph-mon[33312]: osdmap e480: 8 total, 8 up, 8 in 2024-06-18T13:13:23.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:22 smithi179 ceph-mon[33312]: pgmap v720: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:13:23.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:23 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:23 smithi167 ceph-mon[37701]: osdmap e481: 8 total, 8 up, 8 in 2024-06-18T13:13:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:23 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:23 smithi167 ceph-mon[29496]: osdmap e481: 8 total, 8 up, 8 in 2024-06-18T13:13:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:24.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:23 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:24.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:23 smithi179 ceph-mon[33312]: osdmap e481: 8 total, 8 up, 8 in 2024-06-18T13:13:24.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:24.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:24 smithi167 ceph-mon[37701]: osdmap e482: 8 total, 8 up, 8 in 2024-06-18T13:13:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:24 smithi167 ceph-mon[37701]: pgmap v723: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:13:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:24 smithi167 ceph-mon[29496]: osdmap e482: 8 total, 8 up, 8 in 2024-06-18T13:13:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:24 smithi167 ceph-mon[29496]: pgmap v723: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:13:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:24 smithi179 ceph-mon[33312]: osdmap e482: 8 total, 8 up, 8 in 2024-06-18T13:13:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:13:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:24 smithi179 ceph-mon[33312]: pgmap v723: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:13:25.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:13:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:13:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:13:25.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:13:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:25 smithi167 ceph-mon[37701]: osdmap e483: 8 total, 8 up, 8 in 2024-06-18T13:13:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:13:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:13:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:25 smithi167 ceph-mon[29496]: osdmap e483: 8 total, 8 up, 8 in 2024-06-18T13:13:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:13:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:13:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:25 smithi179 ceph-mon[33312]: osdmap e483: 8 total, 8 up, 8 in 2024-06-18T13:13:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:13:26.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:26 smithi167 ceph-mon[37701]: osdmap e484: 8 total, 8 up, 8 in 2024-06-18T13:13:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:26 smithi167 ceph-mon[37701]: pgmap v726: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:13:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:26 smithi167 ceph-mon[29496]: osdmap e484: 8 total, 8 up, 8 in 2024-06-18T13:13:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:26 smithi167 ceph-mon[29496]: pgmap v726: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:13:27.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:26 smithi179 ceph-mon[33312]: osdmap e484: 8 total, 8 up, 8 in 2024-06-18T13:13:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:26 smithi179 ceph-mon[33312]: pgmap v726: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:13:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:27 smithi167 ceph-mon[29496]: osdmap e485: 8 total, 8 up, 8 in 2024-06-18T13:13:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-90"}]: dispatch 2024-06-18T13:13:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-90"}]: dispatch 2024-06-18T13:13:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:27 smithi167 ceph-mon[37701]: osdmap e485: 8 total, 8 up, 8 in 2024-06-18T13:13:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:27.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:27.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-90"}]: dispatch 2024-06-18T13:13:27.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-90"}]: dispatch 2024-06-18T13:13:28.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:27 smithi179 ceph-mon[33312]: osdmap e485: 8 total, 8 up, 8 in 2024-06-18T13:13:28.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:28.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:28.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-90"}]: dispatch 2024-06-18T13:13:28.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-90"}]: dispatch 2024-06-18T13:13:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:28 smithi167 ceph-mon[37701]: osdmap e486: 8 total, 8 up, 8 in 2024-06-18T13:13:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:28 smithi167 ceph-mon[37701]: pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:13:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:28 smithi167 ceph-mon[29496]: osdmap e486: 8 total, 8 up, 8 in 2024-06-18T13:13:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:28 smithi167 ceph-mon[29496]: pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:13:29.018 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:28 smithi179 ceph-mon[33312]: osdmap e486: 8 total, 8 up, 8 in 2024-06-18T13:13:29.019 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:28 smithi179 ceph-mon[33312]: pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:13:29.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:13:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:13:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:29 smithi167 ceph-mon[37701]: osdmap e487: 8 total, 8 up, 8 in 2024-06-18T13:13:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:29 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:29 smithi167 ceph-mon[29496]: osdmap e487: 8 total, 8 up, 8 in 2024-06-18T13:13:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:29 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:30.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:29 smithi179 ceph-mon[33312]: osdmap e487: 8 total, 8 up, 8 in 2024-06-18T13:13:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:29 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:30 smithi167 ceph-mon[37701]: osdmap e488: 8 total, 8 up, 8 in 2024-06-18T13:13:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:30 smithi167 ceph-mon[37701]: pgmap v732: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:13:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:30 smithi167 ceph-mon[29496]: osdmap e488: 8 total, 8 up, 8 in 2024-06-18T13:13:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:30 smithi167 ceph-mon[29496]: pgmap v732: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:13:31.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:30 smithi179 ceph-mon[33312]: osdmap e488: 8 total, 8 up, 8 in 2024-06-18T13:13:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:30 smithi179 ceph-mon[33312]: pgmap v732: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:13:31.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:31 smithi167 ceph-mon[37701]: osdmap e489: 8 total, 8 up, 8 in 2024-06-18T13:13:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:31 smithi167 ceph-mon[29496]: osdmap e489: 8 total, 8 up, 8 in 2024-06-18T13:13:32.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:31 smithi179 ceph-mon[33312]: osdmap e489: 8 total, 8 up, 8 in 2024-06-18T13:13:33.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:32 smithi179 ceph-mon[33312]: osdmap e490: 8 total, 8 up, 8 in 2024-06-18T13:13:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-92"}]: dispatch 2024-06-18T13:13:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-92"}]: dispatch 2024-06-18T13:13:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:32 smithi179 ceph-mon[33312]: pgmap v735: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:13:33.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:32 smithi167 ceph-mon[37701]: osdmap e490: 8 total, 8 up, 8 in 2024-06-18T13:13:33.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:33.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:33.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-92"}]: dispatch 2024-06-18T13:13:33.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-92"}]: dispatch 2024-06-18T13:13:33.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:32 smithi167 ceph-mon[37701]: pgmap v735: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:13:33.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:32 smithi167 ceph-mon[29496]: osdmap e490: 8 total, 8 up, 8 in 2024-06-18T13:13:33.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:33.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:33.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-92"}]: dispatch 2024-06-18T13:13:33.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-92"}]: dispatch 2024-06-18T13:13:33.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:32 smithi167 ceph-mon[29496]: pgmap v735: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-18T13:13:34.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:33 smithi179 ceph-mon[33312]: osdmap e491: 8 total, 8 up, 8 in 2024-06-18T13:13:34.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:33 smithi167 ceph-mon[37701]: osdmap e491: 8 total, 8 up, 8 in 2024-06-18T13:13:34.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:33 smithi167 ceph-mon[29496]: osdmap e491: 8 total, 8 up, 8 in 2024-06-18T13:13:35.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:34 smithi167 ceph-mon[37701]: osdmap e492: 8 total, 8 up, 8 in 2024-06-18T13:13:35.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:35.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:35.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:34 smithi167 ceph-mon[37701]: pgmap v738: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-18T13:13:35.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:34 smithi167 ceph-mon[29496]: osdmap e492: 8 total, 8 up, 8 in 2024-06-18T13:13:35.033 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:35.033 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:35.033 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:34 smithi167 ceph-mon[29496]: pgmap v738: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-18T13:13:35.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:34 smithi179 ceph-mon[33312]: osdmap e492: 8 total, 8 up, 8 in 2024-06-18T13:13:35.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:13:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:34 smithi179 ceph-mon[33312]: pgmap v738: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-18T13:13:35.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:13:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:13:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:13:36.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:35 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:35 smithi179 ceph-mon[33312]: osdmap e493: 8 total, 8 up, 8 in 2024-06-18T13:13:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:36.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:35 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:36.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:36.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:35 smithi167 ceph-mon[37701]: osdmap e493: 8 total, 8 up, 8 in 2024-06-18T13:13:36.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:36.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:35 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:36.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:13:36.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:35 smithi167 ceph-mon[29496]: osdmap e493: 8 total, 8 up, 8 in 2024-06-18T13:13:36.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:13:37.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:36 smithi179 ceph-mon[33312]: osdmap e494: 8 total, 8 up, 8 in 2024-06-18T13:13:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:36 smithi179 ceph-mon[33312]: pgmap v741: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-18T13:13:37.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:36 smithi167 ceph-mon[37701]: osdmap e494: 8 total, 8 up, 8 in 2024-06-18T13:13:37.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:36 smithi167 ceph-mon[37701]: pgmap v741: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-18T13:13:37.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:36 smithi167 ceph-mon[29496]: osdmap e494: 8 total, 8 up, 8 in 2024-06-18T13:13:37.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:36 smithi167 ceph-mon[29496]: pgmap v741: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-18T13:13:38.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:37 smithi179 ceph-mon[33312]: osdmap e495: 8 total, 8 up, 8 in 2024-06-18T13:13:38.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:37 smithi167 ceph-mon[37701]: osdmap e495: 8 total, 8 up, 8 in 2024-06-18T13:13:38.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:37 smithi167 ceph-mon[29496]: osdmap e495: 8 total, 8 up, 8 in 2024-06-18T13:13:39.030 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:38 smithi179 ceph-mon[33312]: pgmap v743: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:13:39.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:38 smithi167 ceph-mon[37701]: pgmap v743: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:13:39.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:38 smithi167 ceph-mon[29496]: pgmap v743: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:13:39.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:13:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:13:40.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:39 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:40.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:39 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:13:40.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:39 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:40.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:39 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:13:40.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:39 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:40.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:39 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:13:41.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:40 smithi179 ceph-mon[33312]: pgmap v744: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-06-18T13:13:41.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:40 smithi167 ceph-mon[37701]: pgmap v744: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-06-18T13:13:41.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:40 smithi167 ceph-mon[29496]: pgmap v744: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-06-18T13:13:42.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:41 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:13:42.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:41 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:13:42.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:41 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:13:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: Adjusting osd_memory_target on smithi167 to 3248M 2024-06-18T13:13:43.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: Adjusting osd_memory_target on smithi179 to 3248M 2024-06-18T13:13:43.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:13:43.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:13:43.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:13:43.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:13:43.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:13:43.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:42 smithi179 ceph-mon[33312]: pgmap v745: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-06-18T13:13:43.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: Adjusting osd_memory_target on smithi167 to 3248M 2024-06-18T13:13:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: Adjusting osd_memory_target on smithi179 to 3248M 2024-06-18T13:13:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:13:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:13:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:13:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:13:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:13:43.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[37701]: pgmap v745: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-06-18T13:13:43.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-18T13:13:43.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: Adjusting osd_memory_target on smithi167 to 3248M 2024-06-18T13:13:43.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: Adjusting osd_memory_target on smithi179 to 3248M 2024-06-18T13:13:43.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:13:43.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:13:43.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:13:43.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:13:43.229 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:13:43.230 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:42 smithi167 ceph-mon[29496]: pgmap v745: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-06-18T13:13:44.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:43 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:44.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:43 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:44.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:43 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:13:45.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:44 smithi167 ceph-mon[29496]: pgmap v746: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 963 B/s wr, 4 op/s 2024-06-18T13:13:45.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:44 smithi167 ceph-mon[37701]: pgmap v746: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 963 B/s wr, 4 op/s 2024-06-18T13:13:45.227 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:13:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:13:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:13:45.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:44 smithi179 ceph-mon[33312]: pgmap v746: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 963 B/s wr, 4 op/s 2024-06-18T13:13:47.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:47 smithi179 ceph-mon[33312]: pgmap v747: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-06-18T13:13:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:47 smithi167 ceph-mon[29496]: pgmap v747: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-06-18T13:13:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:47 smithi167 ceph-mon[37701]: pgmap v747: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-06-18T13:13:48.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:48 smithi179 ceph-mon[33312]: osdmap e496: 8 total, 8 up, 8 in 2024-06-18T13:13:48.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:48 smithi167 ceph-mon[29496]: osdmap e496: 8 total, 8 up, 8 in 2024-06-18T13:13:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:48 smithi167 ceph-mon[37701]: osdmap e496: 8 total, 8 up, 8 in 2024-06-18T13:13:49.325 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:13:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:13:49.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:49 smithi179 ceph-mon[33312]: pgmap v749: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-06-18T13:13:49.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:49 smithi167 ceph-mon[37701]: pgmap v749: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-06-18T13:13:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:49 smithi167 ceph-mon[29496]: pgmap v749: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-06-18T13:13:50.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:50.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:13:51.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:51 smithi179 ceph-mon[33312]: pgmap v750: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:51 smithi167 ceph-mon[29496]: pgmap v750: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:51 smithi167 ceph-mon[37701]: pgmap v750: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:53.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:53 smithi179 ceph-mon[33312]: pgmap v751: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:53.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:53 smithi179 ceph-mon[33312]: osdmap e497: 8 total, 8 up, 8 in 2024-06-18T13:13:53.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:53 smithi167 ceph-mon[37701]: pgmap v751: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:53.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:53 smithi167 ceph-mon[37701]: osdmap e497: 8 total, 8 up, 8 in 2024-06-18T13:13:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:53 smithi167 ceph-mon[29496]: pgmap v751: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:53 smithi167 ceph-mon[29496]: osdmap e497: 8 total, 8 up, 8 in 2024-06-18T13:13:55.340 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:13:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:13:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:13:55.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:55 smithi179 ceph-mon[33312]: pgmap v753: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:13:55.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:55 smithi167 ceph-mon[37701]: pgmap v753: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:13:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:55 smithi167 ceph-mon[29496]: pgmap v753: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:13:57.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:57 smithi179 ceph-mon[33312]: pgmap v754: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:57.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:57 smithi167 ceph-mon[37701]: pgmap v754: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:57.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:57 smithi167 ceph-mon[29496]: pgmap v754: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:58.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:58.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:58.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-94"}]: dispatch 2024-06-18T13:13:58.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-94"}]: dispatch 2024-06-18T13:13:58.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:58.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:58.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-94"}]: dispatch 2024-06-18T13:13:58.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-94"}]: dispatch 2024-06-18T13:13:58.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:58.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:13:58.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-94"}]: dispatch 2024-06-18T13:13:58.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-94"}]: dispatch 2024-06-18T13:13:59.353 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:13:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:13:59.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:59 smithi179 ceph-mon[33312]: pgmap v755: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:59.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:13:59 smithi179 ceph-mon[33312]: osdmap e498: 8 total, 8 up, 8 in 2024-06-18T13:13:59.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:59 smithi167 ceph-mon[29496]: pgmap v755: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:13:59 smithi167 ceph-mon[29496]: osdmap e498: 8 total, 8 up, 8 in 2024-06-18T13:13:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:59 smithi167 ceph-mon[37701]: pgmap v755: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:13:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:13:59 smithi167 ceph-mon[37701]: osdmap e498: 8 total, 8 up, 8 in 2024-06-18T13:14:00.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:00.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:00 smithi179 ceph-mon[33312]: osdmap e499: 8 total, 8 up, 8 in 2024-06-18T13:14:00.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:00.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:00.637 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:14:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=cleanup t=2024-06-18T13:14:00.578037851Z level=info msg="Completed cleanup jobs" duration=1.02926ms 2024-06-18T13:14:00.637 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:14:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=sqlstore.transactions t=2024-06-18T13:14:00.58756251Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-06-18T13:14:00.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:00 smithi167 ceph-mon[37701]: osdmap e499: 8 total, 8 up, 8 in 2024-06-18T13:14:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:00 smithi167 ceph-mon[29496]: osdmap e499: 8 total, 8 up, 8 in 2024-06-18T13:14:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:01.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:01 smithi179 ceph-mon[33312]: pgmap v758: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:01.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:01 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:01.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:01.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:01 smithi179 ceph-mon[33312]: osdmap e500: 8 total, 8 up, 8 in 2024-06-18T13:14:01.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:14:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:01 smithi167 ceph-mon[37701]: pgmap v758: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:01 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:01 smithi167 ceph-mon[37701]: osdmap e500: 8 total, 8 up, 8 in 2024-06-18T13:14:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:14:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:01 smithi167 ceph-mon[29496]: pgmap v758: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:01 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:01.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:01.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:01 smithi167 ceph-mon[29496]: osdmap e500: 8 total, 8 up, 8 in 2024-06-18T13:14:01.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:14:02.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:02 smithi179 ceph-mon[33312]: osdmap e501: 8 total, 8 up, 8 in 2024-06-18T13:14:02.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:02 smithi167 ceph-mon[37701]: osdmap e501: 8 total, 8 up, 8 in 2024-06-18T13:14:02.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:02 smithi167 ceph-mon[29496]: osdmap e501: 8 total, 8 up, 8 in 2024-06-18T13:14:03.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:03 smithi179 ceph-mon[33312]: pgmap v761: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:03 smithi167 ceph-mon[37701]: pgmap v761: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:03.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:03 smithi167 ceph-mon[29496]: pgmap v761: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:05.382 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:14:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:14:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:14:05.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:05 smithi179 ceph-mon[33312]: pgmap v762: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-06-18T13:14:05.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:05 smithi167 ceph-mon[37701]: pgmap v762: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-06-18T13:14:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:05 smithi167 ceph-mon[29496]: pgmap v762: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-06-18T13:14:07.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:07 smithi179 ceph-mon[33312]: pgmap v763: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 741 B/s rd, 741 B/s wr, 2 op/s 2024-06-18T13:14:07.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:07 smithi167 ceph-mon[37701]: pgmap v763: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 741 B/s rd, 741 B/s wr, 2 op/s 2024-06-18T13:14:07.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:07 smithi167 ceph-mon[29496]: pgmap v763: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 741 B/s rd, 741 B/s wr, 2 op/s 2024-06-18T13:14:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:08 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:08.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:08 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:08.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:08 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:09.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:14:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:14:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:09 smithi167 ceph-mon[37701]: pgmap v764: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-06-18T13:14:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.6", "id": [0, 1]}]: dispatch 2024-06-18T13:14:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:09 smithi167 ceph-mon[29496]: pgmap v764: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-06-18T13:14:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.6", "id": [0, 1]}]: dispatch 2024-06-18T13:14:09.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:09 smithi179 ceph-mon[33312]: pgmap v764: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-06-18T13:14:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.6", "id": [0, 1]}]: dispatch 2024-06-18T13:14:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.6", "id": [0, 1]}]': finished 2024-06-18T13:14:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:10 smithi167 ceph-mon[37701]: osdmap e502: 8 total, 8 up, 8 in 2024-06-18T13:14:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:14:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.6", "id": [0, 1]}]': finished 2024-06-18T13:14:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:10 smithi167 ceph-mon[29496]: osdmap e502: 8 total, 8 up, 8 in 2024-06-18T13:14:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:14:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.6", "id": [0, 1]}]': finished 2024-06-18T13:14:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:10 smithi179 ceph-mon[33312]: osdmap e502: 8 total, 8 up, 8 in 2024-06-18T13:14:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:14:11.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:11 smithi167 ceph-mon[37701]: pgmap v766: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 575 B/s wr, 2 op/s 2024-06-18T13:14:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:11 smithi167 ceph-mon[37701]: osdmap e503: 8 total, 8 up, 8 in 2024-06-18T13:14:11.727 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:14:11 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3[58367]: 2024-06-18T13:14:11.397+0000 7f8919e1c640 -1 snap_mapper.add_oid found existing snaps mapped on 106:9866e059:test-rados-api-smithi167-74266-97::foo:21, removing 2024-06-18T13:14:11.727 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:14:11 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0[41556]: 2024-06-18T13:14:11.397+0000 7f1619f16640 -1 snap_mapper.add_oid found existing snaps mapped on 106:9866e059:test-rados-api-smithi167-74266-97::foo:21, removing 2024-06-18T13:14:11.728 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:14:11 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2[52740]: 2024-06-18T13:14:11.397+0000 7f0864c2d640 -1 snap_mapper.add_oid found existing snaps mapped on 106:9866e059:test-rados-api-smithi167-74266-97::foo:21, removing 2024-06-18T13:14:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:11 smithi167 ceph-mon[29496]: pgmap v766: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 575 B/s wr, 2 op/s 2024-06-18T13:14:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:11 smithi167 ceph-mon[29496]: osdmap e503: 8 total, 8 up, 8 in 2024-06-18T13:14:11.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:11 smithi179 ceph-mon[33312]: pgmap v766: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 575 B/s wr, 2 op/s 2024-06-18T13:14:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:11 smithi179 ceph-mon[33312]: osdmap e503: 8 total, 8 up, 8 in 2024-06-18T13:14:12.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:12.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:12.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-96"}]: dispatch 2024-06-18T13:14:12.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-96"}]: dispatch 2024-06-18T13:14:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-96"}]: dispatch 2024-06-18T13:14:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-96"}]: dispatch 2024-06-18T13:14:12.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-96"}]: dispatch 2024-06-18T13:14:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-96"}]: dispatch 2024-06-18T13:14:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:13 smithi167 ceph-mon[37701]: pgmap v768: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:14:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:13 smithi167 ceph-mon[37701]: osdmap e504: 8 total, 8 up, 8 in 2024-06-18T13:14:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:13 smithi167 ceph-mon[29496]: pgmap v768: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:14:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:13 smithi167 ceph-mon[29496]: osdmap e504: 8 total, 8 up, 8 in 2024-06-18T13:14:13.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:13 smithi179 ceph-mon[33312]: pgmap v768: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:14:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:13 smithi179 ceph-mon[33312]: osdmap e504: 8 total, 8 up, 8 in 2024-06-18T13:14:14.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:14 smithi167 ceph-mon[37701]: osdmap e505: 8 total, 8 up, 8 in 2024-06-18T13:14:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:14 smithi167 ceph-mon[29496]: osdmap e505: 8 total, 8 up, 8 in 2024-06-18T13:14:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:14.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:14 smithi179 ceph-mon[33312]: osdmap e505: 8 total, 8 up, 8 in 2024-06-18T13:14:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:15.425 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:14:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:14:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:14:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:15 smithi167 ceph-mon[37701]: pgmap v771: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-18T13:14:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:15 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:15 smithi167 ceph-mon[37701]: osdmap e506: 8 total, 8 up, 8 in 2024-06-18T13:14:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:14:15.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:14:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:15 smithi167 ceph-mon[29496]: pgmap v771: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-18T13:14:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:15 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:15.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:15 smithi167 ceph-mon[29496]: osdmap e506: 8 total, 8 up, 8 in 2024-06-18T13:14:15.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:14:15.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:14:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:15 smithi179 ceph-mon[33312]: pgmap v771: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-18T13:14:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:15 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:15 smithi179 ceph-mon[33312]: osdmap e506: 8 total, 8 up, 8 in 2024-06-18T13:14:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:14:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:14:16.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:14:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:16 smithi167 ceph-mon[37701]: osdmap e507: 8 total, 8 up, 8 in 2024-06-18T13:14:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-98", "mode": "writeback"}]: dispatch 2024-06-18T13:14:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-98", "mode": "writeback"}]: dispatch 2024-06-18T13:14:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:14:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:16 smithi167 ceph-mon[29496]: osdmap e507: 8 total, 8 up, 8 in 2024-06-18T13:14:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-98", "mode": "writeback"}]: dispatch 2024-06-18T13:14:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-98", "mode": "writeback"}]: dispatch 2024-06-18T13:14:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:14:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:16 smithi179 ceph-mon[33312]: osdmap e507: 8 total, 8 up, 8 in 2024-06-18T13:14:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-98", "mode": "writeback"}]: dispatch 2024-06-18T13:14:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-98", "mode": "writeback"}]: dispatch 2024-06-18T13:14:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:17 smithi167 ceph-mon[37701]: pgmap v774: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-06-18T13:14:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:17 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:14:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-98", "mode": "writeback"}]': finished 2024-06-18T13:14:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:17 smithi167 ceph-mon[37701]: osdmap e508: 8 total, 8 up, 8 in 2024-06-18T13:14:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-98"}]: dispatch 2024-06-18T13:14:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-98"}]: dispatch 2024-06-18T13:14:17.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:17 smithi167 ceph-mon[29496]: pgmap v774: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-06-18T13:14:17.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:17 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:14:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-98", "mode": "writeback"}]': finished 2024-06-18T13:14:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:17 smithi167 ceph-mon[29496]: osdmap e508: 8 total, 8 up, 8 in 2024-06-18T13:14:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-98"}]: dispatch 2024-06-18T13:14:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-98"}]: dispatch 2024-06-18T13:14:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:17 smithi179 ceph-mon[33312]: pgmap v774: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-06-18T13:14:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:17 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:14:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-98", "mode": "writeback"}]': finished 2024-06-18T13:14:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:17 smithi179 ceph-mon[33312]: osdmap e508: 8 total, 8 up, 8 in 2024-06-18T13:14:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-98"}]: dispatch 2024-06-18T13:14:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-98"}]: dispatch 2024-06-18T13:14:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-98"}]': finished 2024-06-18T13:14:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:18 smithi167 ceph-mon[37701]: osdmap e509: 8 total, 8 up, 8 in 2024-06-18T13:14:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:14:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:14:18.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:18 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:14:18.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:14:18.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:18 smithi167 ceph-mon[37701]: osdmap e510: 8 total, 8 up, 8 in 2024-06-18T13:14:18.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-98"}]': finished 2024-06-18T13:14:18.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:18 smithi167 ceph-mon[29496]: osdmap e509: 8 total, 8 up, 8 in 2024-06-18T13:14:18.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:14:18.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:14:18.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:18 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:14:18.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:14:18.730 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:18 smithi167 ceph-mon[29496]: osdmap e510: 8 total, 8 up, 8 in 2024-06-18T13:14:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-98"}]': finished 2024-06-18T13:14:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:18 smithi179 ceph-mon[33312]: osdmap e509: 8 total, 8 up, 8 in 2024-06-18T13:14:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:14:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:14:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:18 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:14:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:14:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:18 smithi179 ceph-mon[33312]: osdmap e510: 8 total, 8 up, 8 in 2024-06-18T13:14:19.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:14:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:14:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:19 smithi167 ceph-mon[37701]: pgmap v777: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_count","val": "1"}]: dispatch 2024-06-18T13:14:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_count","val": "1"}]: dispatch 2024-06-18T13:14:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_count","val": "1"}]': finished 2024-06-18T13:14:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:19 smithi167 ceph-mon[37701]: osdmap e511: 8 total, 8 up, 8 in 2024-06-18T13:14:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:19 smithi167 ceph-mon[29496]: pgmap v777: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_count","val": "1"}]: dispatch 2024-06-18T13:14:19.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_count","val": "1"}]: dispatch 2024-06-18T13:14:19.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_count","val": "1"}]': finished 2024-06-18T13:14:19.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:19 smithi167 ceph-mon[29496]: osdmap e511: 8 total, 8 up, 8 in 2024-06-18T13:14:19.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:19 smithi179 ceph-mon[33312]: pgmap v777: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_count","val": "1"}]: dispatch 2024-06-18T13:14:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_count","val": "1"}]: dispatch 2024-06-18T13:14:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_count","val": "1"}]': finished 2024-06-18T13:14:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:19 smithi179 ceph-mon[33312]: osdmap e511: 8 total, 8 up, 8 in 2024-06-18T13:14:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:20 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:14:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:14:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:14:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:20 smithi167 ceph-mon[37701]: osdmap e512: 8 total, 8 up, 8 in 2024-06-18T13:14:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:20 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:14:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:14:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:14:20.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:20 smithi167 ceph-mon[29496]: osdmap e512: 8 total, 8 up, 8 in 2024-06-18T13:14:20.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:20 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:14:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:14:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:14:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:20 smithi179 ceph-mon[33312]: osdmap e512: 8 total, 8 up, 8 in 2024-06-18T13:14:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:21 smithi167 ceph-mon[37701]: pgmap v780: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:14:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "target_max_objects","val": "250"}]: dispatch 2024-06-18T13:14:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "target_max_objects","val": "250"}]: dispatch 2024-06-18T13:14:21.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "target_max_objects","val": "250"}]': finished 2024-06-18T13:14:21.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:21 smithi167 ceph-mon[37701]: osdmap e513: 8 total, 8 up, 8 in 2024-06-18T13:14:21.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:21 smithi167 ceph-mon[29496]: pgmap v780: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:14:21.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "target_max_objects","val": "250"}]: dispatch 2024-06-18T13:14:21.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "target_max_objects","val": "250"}]: dispatch 2024-06-18T13:14:21.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "target_max_objects","val": "250"}]': finished 2024-06-18T13:14:21.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:21 smithi167 ceph-mon[29496]: osdmap e513: 8 total, 8 up, 8 in 2024-06-18T13:14:21.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:21 smithi179 ceph-mon[33312]: pgmap v780: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:14:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "target_max_objects","val": "250"}]: dispatch 2024-06-18T13:14:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "target_max_objects","val": "250"}]: dispatch 2024-06-18T13:14:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-98","var": "target_max_objects","val": "250"}]': finished 2024-06-18T13:14:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:21 smithi179 ceph-mon[33312]: osdmap e513: 8 total, 8 up, 8 in 2024-06-18T13:14:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:22.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:22.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:22.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:22.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:23 smithi167 ceph-mon[37701]: pgmap v783: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:14:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:14:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:23 smithi167 ceph-mon[37701]: osdmap e514: 8 total, 8 up, 8 in 2024-06-18T13:14:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98"}]: dispatch 2024-06-18T13:14:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98"}]: dispatch 2024-06-18T13:14:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:23 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:23 smithi167 ceph-mon[29496]: pgmap v783: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:14:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:14:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:23 smithi167 ceph-mon[29496]: osdmap e514: 8 total, 8 up, 8 in 2024-06-18T13:14:23.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98"}]: dispatch 2024-06-18T13:14:23.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98"}]: dispatch 2024-06-18T13:14:23.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:23 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:23 smithi179 ceph-mon[33312]: pgmap v783: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:14:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:14:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:23 smithi179 ceph-mon[33312]: osdmap e514: 8 total, 8 up, 8 in 2024-06-18T13:14:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98"}]: dispatch 2024-06-18T13:14:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98"}]: dispatch 2024-06-18T13:14:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:23 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98"}]': finished 2024-06-18T13:14:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:24 smithi179 ceph-mon[33312]: osdmap e515: 8 total, 8 up, 8 in 2024-06-18T13:14:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:24 smithi179 ceph-mon[33312]: pgmap v786: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:14:24.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98"}]': finished 2024-06-18T13:14:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:24 smithi167 ceph-mon[37701]: osdmap e515: 8 total, 8 up, 8 in 2024-06-18T13:14:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:24 smithi167 ceph-mon[37701]: pgmap v786: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:14:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-98"}]': finished 2024-06-18T13:14:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:24 smithi167 ceph-mon[29496]: osdmap e515: 8 total, 8 up, 8 in 2024-06-18T13:14:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:24 smithi167 ceph-mon[29496]: pgmap v786: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:14:25.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:14:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:14:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:14:25.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:25 smithi179 ceph-mon[33312]: osdmap e516: 8 total, 8 up, 8 in 2024-06-18T13:14:25.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:14:25.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:25 smithi167 ceph-mon[37701]: osdmap e516: 8 total, 8 up, 8 in 2024-06-18T13:14:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:14:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:25 smithi167 ceph-mon[29496]: osdmap e516: 8 total, 8 up, 8 in 2024-06-18T13:14:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:14:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:26 smithi179 ceph-mon[33312]: osdmap e517: 8 total, 8 up, 8 in 2024-06-18T13:14:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:26 smithi179 ceph-mon[33312]: pgmap v789: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:26 smithi179 ceph-mon[33312]: osdmap e518: 8 total, 8 up, 8 in 2024-06-18T13:14:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:14:26.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:14:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[37701]: osdmap e517: 8 total, 8 up, 8 in 2024-06-18T13:14:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[37701]: pgmap v789: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[37701]: osdmap e518: 8 total, 8 up, 8 in 2024-06-18T13:14:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:14:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:14:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[29496]: osdmap e517: 8 total, 8 up, 8 in 2024-06-18T13:14:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[29496]: pgmap v789: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[29496]: osdmap e518: 8 total, 8 up, 8 in 2024-06-18T13:14:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:14:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:14:28.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:14:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:28 smithi179 ceph-mon[33312]: osdmap e519: 8 total, 8 up, 8 in 2024-06-18T13:14:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-100"}]: dispatch 2024-06-18T13:14:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-100"}]: dispatch 2024-06-18T13:14:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:28 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:28 smithi179 ceph-mon[33312]: pgmap v792: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:14:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:28 smithi167 ceph-mon[37701]: osdmap e519: 8 total, 8 up, 8 in 2024-06-18T13:14:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-100"}]: dispatch 2024-06-18T13:14:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-100"}]: dispatch 2024-06-18T13:14:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:28 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:28 smithi167 ceph-mon[37701]: pgmap v792: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:14:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:28 smithi167 ceph-mon[29496]: osdmap e519: 8 total, 8 up, 8 in 2024-06-18T13:14:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-100"}]: dispatch 2024-06-18T13:14:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-100"}]: dispatch 2024-06-18T13:14:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:28 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:28 smithi167 ceph-mon[29496]: pgmap v792: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:29.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:14:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:14:29.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-100"}]': finished 2024-06-18T13:14:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:29 smithi179 ceph-mon[33312]: osdmap e520: 8 total, 8 up, 8 in 2024-06-18T13:14:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-100", "mode": "writeback"}]: dispatch 2024-06-18T13:14:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-100", "mode": "writeback"}]: dispatch 2024-06-18T13:14:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:29 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:29.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-100"}]': finished 2024-06-18T13:14:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:29 smithi167 ceph-mon[37701]: osdmap e520: 8 total, 8 up, 8 in 2024-06-18T13:14:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-100", "mode": "writeback"}]: dispatch 2024-06-18T13:14:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-100", "mode": "writeback"}]: dispatch 2024-06-18T13:14:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:29 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-6", "overlaypool": "test-rados-api-smithi167-74266-100"}]': finished 2024-06-18T13:14:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:29 smithi167 ceph-mon[29496]: osdmap e520: 8 total, 8 up, 8 in 2024-06-18T13:14:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-100", "mode": "writeback"}]: dispatch 2024-06-18T13:14:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-100", "mode": "writeback"}]: dispatch 2024-06-18T13:14:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:29 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:30.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:30 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:14:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-100", "mode": "writeback"}]': finished 2024-06-18T13:14:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:30 smithi179 ceph-mon[33312]: osdmap e521: 8 total, 8 up, 8 in 2024-06-18T13:14:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:14:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:14:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:30 smithi179 ceph-mon[33312]: pgmap v795: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:30 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:14:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-100", "mode": "writeback"}]': finished 2024-06-18T13:14:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:30 smithi167 ceph-mon[37701]: osdmap e521: 8 total, 8 up, 8 in 2024-06-18T13:14:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:14:30.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:14:30.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:30 smithi167 ceph-mon[37701]: pgmap v795: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:30 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:14:30.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-100", "mode": "writeback"}]': finished 2024-06-18T13:14:30.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:30 smithi167 ceph-mon[29496]: osdmap e521: 8 total, 8 up, 8 in 2024-06-18T13:14:30.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:14:30.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:14:30.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:30 smithi167 ceph-mon[29496]: pgmap v795: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:31.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:14:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:31 smithi179 ceph-mon[33312]: osdmap e522: 8 total, 8 up, 8 in 2024-06-18T13:14:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:14:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:14:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:14:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:31 smithi167 ceph-mon[37701]: osdmap e522: 8 total, 8 up, 8 in 2024-06-18T13:14:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:14:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:14:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:14:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:31 smithi167 ceph-mon[29496]: osdmap e522: 8 total, 8 up, 8 in 2024-06-18T13:14:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:14:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:14:32.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:14:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:32 smithi179 ceph-mon[33312]: osdmap e523: 8 total, 8 up, 8 in 2024-06-18T13:14:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:14:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:14:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:32 smithi179 ceph-mon[33312]: pgmap v798: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:14:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:32 smithi167 ceph-mon[37701]: osdmap e523: 8 total, 8 up, 8 in 2024-06-18T13:14:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:14:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:14:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:32 smithi167 ceph-mon[37701]: pgmap v798: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:14:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:32 smithi167 ceph-mon[29496]: osdmap e523: 8 total, 8 up, 8 in 2024-06-18T13:14:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:14:32.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:14:32.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:32 smithi167 ceph-mon[29496]: pgmap v798: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:33 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:14:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:33 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-18T13:14:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:33 smithi179 ceph-mon[33312]: osdmap e524: 8 total, 8 up, 8 in 2024-06-18T13:14:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-06-18T13:14:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:33 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-06-18T13:14:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:33 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:33.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:33 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:14:33.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:33 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-18T13:14:33.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:33 smithi167 ceph-mon[37701]: osdmap e524: 8 total, 8 up, 8 in 2024-06-18T13:14:33.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-06-18T13:14:33.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:33 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-06-18T13:14:33.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:33 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:33.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:33 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:14:33.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:33 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-18T13:14:33.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:33 smithi167 ceph-mon[29496]: osdmap e524: 8 total, 8 up, 8 in 2024-06-18T13:14:33.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-06-18T13:14:33.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:33 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-06-18T13:14:33.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:33 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-06-18T13:14:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:34 smithi179 ceph-mon[33312]: osdmap e525: 8 total, 8 up, 8 in 2024-06-18T13:14:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:34 smithi179 ceph-mon[33312]: pgmap v801: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:14:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-06-18T13:14:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:34 smithi167 ceph-mon[37701]: osdmap e525: 8 total, 8 up, 8 in 2024-06-18T13:14:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:34 smithi167 ceph-mon[37701]: pgmap v801: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:14:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-06-18T13:14:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:34 smithi167 ceph-mon[29496]: osdmap e525: 8 total, 8 up, 8 in 2024-06-18T13:14:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:34 smithi167 ceph-mon[29496]: pgmap v801: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:14:35.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:14:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:14:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:14:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:14:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:35 smithi179 ceph-mon[33312]: osdmap e526: 8 total, 8 up, 8 in 2024-06-18T13:14:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100"}]: dispatch 2024-06-18T13:14:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100"}]: dispatch 2024-06-18T13:14:35.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:14:35.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:35 smithi167 ceph-mon[37701]: osdmap e526: 8 total, 8 up, 8 in 2024-06-18T13:14:35.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100"}]: dispatch 2024-06-18T13:14:35.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100"}]: dispatch 2024-06-18T13:14:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]': finished 2024-06-18T13:14:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:35 smithi167 ceph-mon[29496]: osdmap e526: 8 total, 8 up, 8 in 2024-06-18T13:14:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100"}]: dispatch 2024-06-18T13:14:35.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100"}]: dispatch 2024-06-18T13:14:36.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100"}]': finished 2024-06-18T13:14:36.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:36 smithi179 ceph-mon[33312]: osdmap e527: 8 total, 8 up, 8 in 2024-06-18T13:14:36.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:36 smithi179 ceph-mon[33312]: pgmap v804: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:14:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100"}]': finished 2024-06-18T13:14:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:36 smithi167 ceph-mon[29496]: osdmap e527: 8 total, 8 up, 8 in 2024-06-18T13:14:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:36 smithi167 ceph-mon[29496]: pgmap v804: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:14:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-100"}]': finished 2024-06-18T13:14:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:36 smithi167 ceph-mon[37701]: osdmap e527: 8 total, 8 up, 8 in 2024-06-18T13:14:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:36 smithi167 ceph-mon[37701]: pgmap v804: 260 pgs: 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-18T13:14:37.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:37 smithi179 ceph-mon[33312]: osdmap e528: 8 total, 8 up, 8 in 2024-06-18T13:14:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:37 smithi167 ceph-mon[29496]: osdmap e528: 8 total, 8 up, 8 in 2024-06-18T13:14:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:37 smithi167 ceph-mon[37701]: osdmap e528: 8 total, 8 up, 8 in 2024-06-18T13:14:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:38 smithi179 ceph-mon[33312]: osdmap e529: 8 total, 8 up, 8 in 2024-06-18T13:14:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:38 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:38 smithi179 ceph-mon[33312]: pgmap v807: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:38 smithi179 ceph-mon[33312]: osdmap e530: 8 total, 8 up, 8 in 2024-06-18T13:14:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:38 smithi167 ceph-mon[29496]: osdmap e529: 8 total, 8 up, 8 in 2024-06-18T13:14:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:38 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:38 smithi167 ceph-mon[29496]: pgmap v807: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:38 smithi167 ceph-mon[29496]: osdmap e530: 8 total, 8 up, 8 in 2024-06-18T13:14:38.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:38 smithi167 ceph-mon[37701]: osdmap e529: 8 total, 8 up, 8 in 2024-06-18T13:14:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:38 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:38 smithi167 ceph-mon[37701]: pgmap v807: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:38 smithi167 ceph-mon[37701]: osdmap e530: 8 total, 8 up, 8 in 2024-06-18T13:14:39.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:14:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:14:39.585 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:14:39.585 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:39.586 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:14:39.586 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:14:39.586 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:39.586 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:14:39.586 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[29496]: osdmap e531: 8 total, 8 up, 8 in 2024-06-18T13:14:39.586 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:14:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:14:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:39 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:14:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:39 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:14:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:39 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:39 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:14:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:39 smithi179 ceph-mon[33312]: osdmap e531: 8 total, 8 up, 8 in 2024-06-18T13:14:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:14:39.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:39 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:14:39.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:14:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-18T13:14:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:14:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-18T13:14:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-18T13:14:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[37701]: osdmap e531: 8 total, 8 up, 8 in 2024-06-18T13:14:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:14:39.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:39 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-18T13:14:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:14:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:40 smithi179 ceph-mon[33312]: pgmap v810: 260 pgs: 9 creating+activating, 7 creating+peering, 244 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:14:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:40 smithi179 ceph-mon[33312]: osdmap e532: 8 total, 8 up, 8 in 2024-06-18T13:14:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:14:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:14:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:14:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:40 smithi167 ceph-mon[29496]: pgmap v810: 260 pgs: 9 creating+activating, 7 creating+peering, 244 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:14:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:40 smithi167 ceph-mon[29496]: osdmap e532: 8 total, 8 up, 8 in 2024-06-18T13:14:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:14:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:14:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:14:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:40 smithi167 ceph-mon[37701]: pgmap v810: 260 pgs: 9 creating+activating, 7 creating+peering, 244 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:40.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-18T13:14:40.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:40 smithi167 ceph-mon[37701]: osdmap e532: 8 total, 8 up, 8 in 2024-06-18T13:14:40.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:14:40.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-18T13:14:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:14:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:42 smithi179 ceph-mon[33312]: osdmap e533: 8 total, 8 up, 8 in 2024-06-18T13:14:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-102"}]: dispatch 2024-06-18T13:14:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-102"}]: dispatch 2024-06-18T13:14:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:14:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:42 smithi179 ceph-mon[33312]: pgmap v813: 260 pgs: 9 creating+activating, 7 creating+peering, 244 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:14:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:14:42.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:14:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:14:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[37701]: osdmap e533: 8 total, 8 up, 8 in 2024-06-18T13:14:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-102"}]: dispatch 2024-06-18T13:14:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-102"}]: dispatch 2024-06-18T13:14:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:14:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[37701]: pgmap v813: 260 pgs: 9 creating+activating, 7 creating+peering, 244 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:42.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:14:42.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:14:42.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:14:42.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-18T13:14:42.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[29496]: osdmap e533: 8 total, 8 up, 8 in 2024-06-18T13:14:42.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:42.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-6"}]: dispatch 2024-06-18T13:14:42.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1661886232' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-102"}]: dispatch 2024-06-18T13:14:42.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-6", "tierpool": "test-rados-api-smithi167-74266-102"}]: dispatch 2024-06-18T13:14:42.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:14:42.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[29496]: pgmap v813: 260 pgs: 9 creating+activating, 7 creating+peering, 244 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:42.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:14:42.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:14:42.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:14:43.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:43 smithi179 ceph-mon[33312]: osdmap e534: 8 total, 8 up, 8 in 2024-06-18T13:14:43.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:43 smithi167 ceph-mon[29496]: osdmap e534: 8 total, 8 up, 8 in 2024-06-18T13:14:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:43 smithi167 ceph-mon[37701]: osdmap e534: 8 total, 8 up, 8 in 2024-06-18T13:14:44.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:44 smithi179 ceph-mon[33312]: osdmap e535: 8 total, 8 up, 8 in 2024-06-18T13:14:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:14:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:14:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:14:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:44 smithi179 ceph-mon[33312]: pgmap v816: 196 pgs: 196 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:44 smithi167 ceph-mon[37701]: osdmap e535: 8 total, 8 up, 8 in 2024-06-18T13:14:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:14:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:14:44.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:14:44.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:44 smithi167 ceph-mon[37701]: pgmap v816: 196 pgs: 196 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:44.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:44 smithi167 ceph-mon[29496]: osdmap e535: 8 total, 8 up, 8 in 2024-06-18T13:14:44.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:14:44.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:14:44.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:14:44.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:44 smithi167 ceph-mon[29496]: pgmap v816: 196 pgs: 196 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:14:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:14:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:14:46.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:14:46.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:45 smithi179 ceph-mon[33312]: osdmap e536: 8 total, 8 up, 8 in 2024-06-18T13:14:46.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi167-74266-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:14:46.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:14:46.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:45 smithi167 ceph-mon[37701]: osdmap e536: 8 total, 8 up, 8 in 2024-06-18T13:14:46.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi167-74266-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:14:46.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:14:46.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:45 smithi167 ceph-mon[29496]: osdmap e536: 8 total, 8 up, 8 in 2024-06-18T13:14:46.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi167-74266-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:14:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:46 smithi179 ceph-mon[33312]: osdmap e537: 8 total, 8 up, 8 in 2024-06-18T13:14:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:46 smithi179 ceph-mon[33312]: pgmap v819: 196 pgs: 196 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi167-74266-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi167-74266-104"}]': finished 2024-06-18T13:14:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:46 smithi179 ceph-mon[33312]: osdmap e538: 8 total, 8 up, 8 in 2024-06-18T13:14:47.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:46 smithi167 ceph-mon[37701]: osdmap e537: 8 total, 8 up, 8 in 2024-06-18T13:14:47.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:46 smithi167 ceph-mon[37701]: pgmap v819: 196 pgs: 196 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:47.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi167-74266-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi167-74266-104"}]': finished 2024-06-18T13:14:47.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:46 smithi167 ceph-mon[37701]: osdmap e538: 8 total, 8 up, 8 in 2024-06-18T13:14:47.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:46 smithi167 ceph-mon[29496]: osdmap e537: 8 total, 8 up, 8 in 2024-06-18T13:14:47.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:46 smithi167 ceph-mon[29496]: pgmap v819: 196 pgs: 196 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:47.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi167-74266-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi167-74266-104"}]': finished 2024-06-18T13:14:47.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:46 smithi167 ceph-mon[29496]: osdmap e538: 8 total, 8 up, 8 in 2024-06-18T13:14:48.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:48 smithi179 ceph-mon[33312]: osdmap e539: 8 total, 8 up, 8 in 2024-06-18T13:14:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:48 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:48 smithi179 ceph-mon[33312]: pgmap v822: 204 pgs: 8 unknown, 196 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:48 smithi167 ceph-mon[37701]: osdmap e539: 8 total, 8 up, 8 in 2024-06-18T13:14:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:48 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:48 smithi167 ceph-mon[37701]: pgmap v822: 204 pgs: 8 unknown, 196 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:48 smithi167 ceph-mon[29496]: osdmap e539: 8 total, 8 up, 8 in 2024-06-18T13:14:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:48 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:48 smithi167 ceph-mon[29496]: pgmap v822: 204 pgs: 8 unknown, 196 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:49.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:14:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:14:49.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:49 smithi179 ceph-mon[33312]: osdmap e540: 8 total, 8 up, 8 in 2024-06-18T13:14:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:49 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:49 smithi167 ceph-mon[37701]: osdmap e540: 8 total, 8 up, 8 in 2024-06-18T13:14:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:49 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:49 smithi167 ceph-mon[29496]: osdmap e540: 8 total, 8 up, 8 in 2024-06-18T13:14:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:49 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:50.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:50 smithi179 ceph-mon[33312]: osdmap e541: 8 total, 8 up, 8 in 2024-06-18T13:14:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:50 smithi179 ceph-mon[33312]: pgmap v825: 236 pgs: 1 creating+peering, 31 unknown, 204 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:14:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:50 smithi167 ceph-mon[37701]: osdmap e541: 8 total, 8 up, 8 in 2024-06-18T13:14:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:50 smithi167 ceph-mon[37701]: pgmap v825: 236 pgs: 1 creating+peering, 31 unknown, 204 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:14:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:50 smithi167 ceph-mon[29496]: osdmap e541: 8 total, 8 up, 8 in 2024-06-18T13:14:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:50 smithi167 ceph-mon[29496]: pgmap v825: 236 pgs: 1 creating+peering, 31 unknown, 204 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:14:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:51 smithi167 ceph-mon[37701]: osdmap e542: 8 total, 8 up, 8 in 2024-06-18T13:14:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:51 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:51 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:51 smithi167 ceph-mon[29496]: osdmap e542: 8 total, 8 up, 8 in 2024-06-18T13:14:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:51 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:51 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:52.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:51 smithi179 ceph-mon[33312]: osdmap e542: 8 total, 8 up, 8 in 2024-06-18T13:14:52.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:51 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:51 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:14:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:52 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:52 smithi167 ceph-mon[37701]: osdmap e543: 8 total, 8 up, 8 in 2024-06-18T13:14:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:52 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:52 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:52 smithi167 ceph-mon[37701]: pgmap v828: 268 pgs: 1 creating+peering, 63 unknown, 204 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:14:52.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:52 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:52.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:52 smithi167 ceph-mon[29496]: osdmap e543: 8 total, 8 up, 8 in 2024-06-18T13:14:52.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:52 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:52.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:52 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:52.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:52 smithi167 ceph-mon[29496]: pgmap v828: 268 pgs: 1 creating+peering, 63 unknown, 204 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:14:53.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:52 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:14:53.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:52 smithi179 ceph-mon[33312]: osdmap e543: 8 total, 8 up, 8 in 2024-06-18T13:14:53.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:52 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:53.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:52 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:53.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:52 smithi179 ceph-mon[33312]: pgmap v828: 268 pgs: 1 creating+peering, 63 unknown, 204 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:14:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]': finished 2024-06-18T13:14:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:53 smithi167 ceph-mon[37701]: osdmap e544: 8 total, 8 up, 8 in 2024-06-18T13:14:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:53 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-107", "overlaypool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:53 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-107", "overlaypool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:53 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:53.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]': finished 2024-06-18T13:14:53.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:53 smithi167 ceph-mon[29496]: osdmap e544: 8 total, 8 up, 8 in 2024-06-18T13:14:53.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:53 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-107", "overlaypool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:53.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:53 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-107", "overlaypool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:53.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:53 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:54.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]': finished 2024-06-18T13:14:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:53 smithi179 ceph-mon[33312]: osdmap e544: 8 total, 8 up, 8 in 2024-06-18T13:14:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:53 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-107", "overlaypool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:53 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-107", "overlaypool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:53 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:14:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-107", "overlaypool": "test-rados-api-smithi167-74266-107-cache"}]': finished 2024-06-18T13:14:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:54 smithi167 ceph-mon[29496]: osdmap e545: 8 total, 8 up, 8 in 2024-06-18T13:14:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:54 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-107-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:14:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:54 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-107-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:14:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:54 smithi167 ceph-mon[29496]: pgmap v831: 268 pgs: 268 active+clean; 583 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-107", "overlaypool": "test-rados-api-smithi167-74266-107-cache"}]': finished 2024-06-18T13:14:54.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:54 smithi167 ceph-mon[37701]: osdmap e545: 8 total, 8 up, 8 in 2024-06-18T13:14:54.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:54 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-107-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:14:54.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:54 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-107-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:14:54.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:54 smithi167 ceph-mon[37701]: pgmap v831: 268 pgs: 268 active+clean; 583 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:55.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-107", "overlaypool": "test-rados-api-smithi167-74266-107-cache"}]': finished 2024-06-18T13:14:55.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:54 smithi179 ceph-mon[33312]: osdmap e545: 8 total, 8 up, 8 in 2024-06-18T13:14:55.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:54 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-107-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:14:55.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:54 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-107-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:14:55.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:54 smithi179 ceph-mon[33312]: pgmap v831: 268 pgs: 268 active+clean; 583 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:55.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:14:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:14:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:14:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-107-cache", "mode": "writeback"}]': finished 2024-06-18T13:14:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-mon[37701]: osdmap e546: 8 total, 8 up, 8 in 2024-06-18T13:14:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-107"}]: dispatch 2024-06-18T13:14:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-107"}]: dispatch 2024-06-18T13:14:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:14:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:14:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:14:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-107-cache", "mode": "writeback"}]': finished 2024-06-18T13:14:55.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-mon[29496]: osdmap e546: 8 total, 8 up, 8 in 2024-06-18T13:14:55.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-107"}]: dispatch 2024-06-18T13:14:55.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-107"}]: dispatch 2024-06-18T13:14:55.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:14:55.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:14:56.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:55 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:14:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-107-cache", "mode": "writeback"}]': finished 2024-06-18T13:14:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:55 smithi179 ceph-mon[33312]: osdmap e546: 8 total, 8 up, 8 in 2024-06-18T13:14:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:55 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-107"}]: dispatch 2024-06-18T13:14:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:55 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-107"}]: dispatch 2024-06-18T13:14:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:14:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:14:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-107"}]': finished 2024-06-18T13:14:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[37701]: osdmap e547: 8 total, 8 up, 8 in 2024-06-18T13:14:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[37701]: pgmap v834: 268 pgs: 268 active+clean; 583 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:14:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]': finished 2024-06-18T13:14:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[37701]: osdmap e548: 8 total, 8 up, 8 in 2024-06-18T13:14:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-107"}]': finished 2024-06-18T13:14:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[29496]: osdmap e547: 8 total, 8 up, 8 in 2024-06-18T13:14:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[29496]: pgmap v834: 268 pgs: 268 active+clean; 583 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:14:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]': finished 2024-06-18T13:14:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:56 smithi167 ceph-mon[29496]: osdmap e548: 8 total, 8 up, 8 in 2024-06-18T13:14:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-107"}]': finished 2024-06-18T13:14:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:56 smithi179 ceph-mon[33312]: osdmap e547: 8 total, 8 up, 8 in 2024-06-18T13:14:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/4242659159' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]: dispatch 2024-06-18T13:14:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:56 smithi179 ceph-mon[33312]: pgmap v834: 268 pgs: 268 active+clean; 583 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:14:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:56 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:14:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-107", "tierpool": "test-rados-api-smithi167-74266-107-cache"}]': finished 2024-06-18T13:14:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:56 smithi179 ceph-mon[33312]: osdmap e548: 8 total, 8 up, 8 in 2024-06-18T13:14:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:58 smithi167 ceph-mon[37701]: osdmap e549: 8 total, 8 up, 8 in 2024-06-18T13:14:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:58 smithi167 ceph-mon[37701]: pgmap v837: 236 pgs: 236 active+clean; 583 KiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:58.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:58 smithi167 ceph-mon[29496]: osdmap e549: 8 total, 8 up, 8 in 2024-06-18T13:14:58.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:58 smithi167 ceph-mon[29496]: pgmap v837: 236 pgs: 236 active+clean; 583 KiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:59.119 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:58 smithi179 ceph-mon[33312]: osdmap e549: 8 total, 8 up, 8 in 2024-06-18T13:14:59.119 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:58 smithi179 ceph-mon[33312]: pgmap v837: 236 pgs: 236 active+clean; 583 KiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:14:59.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:14:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:14:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[37701]: osdmap e550: 8 total, 8 up, 8 in 2024-06-18T13:14:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:14:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:14:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:14:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:14:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:14:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:14:59.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:14:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[29496]: osdmap e550: 8 total, 8 up, 8 in 2024-06-18T13:14:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:14:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:14:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:14:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:14:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:14:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:14:59.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:14:59 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:00.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:59 smithi179 ceph-mon[33312]: osdmap e550: 8 total, 8 up, 8 in 2024-06-18T13:15:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:59 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:59 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:59 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:15:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:15:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:14:59 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:15:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:00 smithi167 ceph-mon[37701]: osdmap e551: 8 total, 8 up, 8 in 2024-06-18T13:15:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:00 smithi167 ceph-mon[37701]: pgmap v840: 204 pgs: 204 active+clean; 583 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:15:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:00 smithi167 ceph-mon[29496]: osdmap e551: 8 total, 8 up, 8 in 2024-06-18T13:15:01.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:01.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:01.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:00 smithi167 ceph-mon[29496]: pgmap v840: 204 pgs: 204 active+clean; 583 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:15:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:00 smithi179 ceph-mon[33312]: osdmap e551: 8 total, 8 up, 8 in 2024-06-18T13:15:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:00 smithi179 ceph-mon[33312]: pgmap v840: 204 pgs: 204 active+clean; 583 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:02.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:01 smithi167 ceph-mon[37701]: osdmap e552: 8 total, 8 up, 8 in 2024-06-18T13:15:02.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-109"}]': finished 2024-06-18T13:15:02.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:01 smithi167 ceph-mon[37701]: osdmap e553: 8 total, 8 up, 8 in 2024-06-18T13:15:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:01 smithi167 ceph-mon[29496]: osdmap e552: 8 total, 8 up, 8 in 2024-06-18T13:15:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-109"}]': finished 2024-06-18T13:15:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:01 smithi167 ceph-mon[29496]: osdmap e553: 8 total, 8 up, 8 in 2024-06-18T13:15:02.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:01 smithi179 ceph-mon[33312]: osdmap e552: 8 total, 8 up, 8 in 2024-06-18T13:15:02.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-109"}]': finished 2024-06-18T13:15:02.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:01 smithi179 ceph-mon[33312]: osdmap e553: 8 total, 8 up, 8 in 2024-06-18T13:15:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:02 smithi167 ceph-mon[29496]: pgmap v843: 212 pgs: 8 unknown, 204 active+clean; 583 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:02 smithi167 ceph-mon[29496]: osdmap e554: 8 total, 8 up, 8 in 2024-06-18T13:15:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:02 smithi167 ceph-mon[37701]: pgmap v843: 212 pgs: 8 unknown, 204 active+clean; 583 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:02 smithi167 ceph-mon[37701]: osdmap e554: 8 total, 8 up, 8 in 2024-06-18T13:15:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:02 smithi179 ceph-mon[33312]: pgmap v843: 212 pgs: 8 unknown, 204 active+clean; 583 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:02 smithi179 ceph-mon[33312]: osdmap e554: 8 total, 8 up, 8 in 2024-06-18T13:15:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:04.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:03 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:15:04.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:03 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:15:04.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:03 smithi167 ceph-mon[29496]: osdmap e555: 8 total, 8 up, 8 in 2024-06-18T13:15:04.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:03 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:04.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:03 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:04.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:03 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:15:04.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:03 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:15:04.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:03 smithi167 ceph-mon[37701]: osdmap e555: 8 total, 8 up, 8 in 2024-06-18T13:15:04.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:03 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:04.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:03 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:04.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:03 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:15:04.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:03 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:15:04.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:03 smithi179 ceph-mon[33312]: osdmap e555: 8 total, 8 up, 8 in 2024-06-18T13:15:04.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:03 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:04.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:03 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:05.226 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:15:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:15:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:15:05.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:04 smithi167 ceph-mon[37701]: pgmap v846: 244 pgs: 32 creating+peering, 212 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:05.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]': finished 2024-06-18T13:15:05.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:04 smithi167 ceph-mon[37701]: osdmap e556: 8 total, 8 up, 8 in 2024-06-18T13:15:05.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-109", "overlaypool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:05.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-109", "overlaypool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:05.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:04 smithi167 ceph-mon[29496]: pgmap v846: 244 pgs: 32 creating+peering, 212 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:05.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]': finished 2024-06-18T13:15:05.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:04 smithi167 ceph-mon[29496]: osdmap e556: 8 total, 8 up, 8 in 2024-06-18T13:15:05.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-109", "overlaypool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:05.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-109", "overlaypool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:05.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:04 smithi179 ceph-mon[33312]: pgmap v846: 244 pgs: 32 creating+peering, 212 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:05.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]': finished 2024-06-18T13:15:05.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:04 smithi179 ceph-mon[33312]: osdmap e556: 8 total, 8 up, 8 in 2024-06-18T13:15:05.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-109", "overlaypool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:05.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-109", "overlaypool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-109", "overlaypool": "test-rados-api-smithi167-74266-109-cache"}]': finished 2024-06-18T13:15:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:06 smithi167 ceph-mon[37701]: osdmap e557: 8 total, 8 up, 8 in 2024-06-18T13:15:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-109-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:15:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-109-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:15:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:06 smithi167 ceph-mon[37701]: pgmap v849: 244 pgs: 32 creating+peering, 212 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:06.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-109", "overlaypool": "test-rados-api-smithi167-74266-109-cache"}]': finished 2024-06-18T13:15:06.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:06 smithi167 ceph-mon[29496]: osdmap e557: 8 total, 8 up, 8 in 2024-06-18T13:15:06.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-109-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:15:06.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-109-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:15:06.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:06 smithi167 ceph-mon[29496]: pgmap v849: 244 pgs: 32 creating+peering, 212 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-109", "overlaypool": "test-rados-api-smithi167-74266-109-cache"}]': finished 2024-06-18T13:15:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:06 smithi179 ceph-mon[33312]: osdmap e557: 8 total, 8 up, 8 in 2024-06-18T13:15:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-109-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:15:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-109-cache", "mode": "writeback"}]: dispatch 2024-06-18T13:15:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:06 smithi179 ceph-mon[33312]: pgmap v849: 244 pgs: 32 creating+peering, 212 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:07 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:15:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-109-cache", "mode": "writeback"}]': finished 2024-06-18T13:15:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:07 smithi167 ceph-mon[37701]: osdmap e558: 8 total, 8 up, 8 in 2024-06-18T13:15:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:15:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:15:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:07 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:15:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-109-cache", "mode": "writeback"}]': finished 2024-06-18T13:15:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:07 smithi167 ceph-mon[29496]: osdmap e558: 8 total, 8 up, 8 in 2024-06-18T13:15:07.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:15:07.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:15:08.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:07 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:15:08.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-109-cache", "mode": "writeback"}]': finished 2024-06-18T13:15:08.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:07 smithi179 ceph-mon[33312]: osdmap e558: 8 total, 8 up, 8 in 2024-06-18T13:15:08.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:15:08.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:15:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:15:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:08 smithi167 ceph-mon[37701]: osdmap e559: 8 total, 8 up, 8 in 2024-06-18T13:15:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:15:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:15:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:08 smithi167 ceph-mon[37701]: pgmap v852: 244 pgs: 32 creating+peering, 212 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:15:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:15:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:15:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:08 smithi167 ceph-mon[29496]: osdmap e559: 8 total, 8 up, 8 in 2024-06-18T13:15:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:15:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:15:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:08 smithi167 ceph-mon[29496]: pgmap v852: 244 pgs: 32 creating+peering, 212 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:15:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:15:09.130 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:15:09.130 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:08 smithi179 ceph-mon[33312]: osdmap e559: 8 total, 8 up, 8 in 2024-06-18T13:15:09.130 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:15:09.130 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:15:09.131 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:08 smithi179 ceph-mon[33312]: pgmap v852: 244 pgs: 32 creating+peering, 212 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:15:09.131 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:15:09.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:15:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: osdmap e560: 8 total, 8 up, 8 in 2024-06-18T13:15:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:15:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:15:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.9", "id": [2, 3]}]: dispatch 2024-06-18T13:15:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.11", "id": [2, 3]}]: dispatch 2024-06-18T13:15:09.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.13", "id": [5, 3]}]: dispatch 2024-06-18T13:15:09.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.1e", "id": [5, 4]}]: dispatch 2024-06-18T13:15:09.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:15:09.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-18T13:15:09.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.9", "id": [2, 3]}]': finished 2024-06-18T13:15:09.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.11", "id": [2, 3]}]': finished 2024-06-18T13:15:09.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.13", "id": [5, 3]}]': finished 2024-06-18T13:15:09.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.1e", "id": [5, 4]}]': finished 2024-06-18T13:15:09.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: osdmap e561: 8 total, 8 up, 8 in 2024-06-18T13:15:09.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-06-18T13:15:09.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-06-18T13:15:09.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: osdmap e560: 8 total, 8 up, 8 in 2024-06-18T13:15:09.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:15:09.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:15:09.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:09.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.9", "id": [2, 3]}]: dispatch 2024-06-18T13:15:09.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.11", "id": [2, 3]}]: dispatch 2024-06-18T13:15:09.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.13", "id": [5, 3]}]: dispatch 2024-06-18T13:15:09.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.1e", "id": [5, 4]}]: dispatch 2024-06-18T13:15:09.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:15:09.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-18T13:15:09.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.9", "id": [2, 3]}]': finished 2024-06-18T13:15:09.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.11", "id": [2, 3]}]': finished 2024-06-18T13:15:09.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.13", "id": [5, 3]}]': finished 2024-06-18T13:15:09.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.1e", "id": [5, 4]}]': finished 2024-06-18T13:15:09.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: osdmap e561: 8 total, 8 up, 8 in 2024-06-18T13:15:09.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-06-18T13:15:09.980 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-06-18T13:15:10.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: osdmap e560: 8 total, 8 up, 8 in 2024-06-18T13:15:10.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:15:10.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:15:10.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:10.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.9", "id": [2, 3]}]: dispatch 2024-06-18T13:15:10.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.11", "id": [2, 3]}]: dispatch 2024-06-18T13:15:10.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.13", "id": [5, 3]}]: dispatch 2024-06-18T13:15:10.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.1e", "id": [5, 4]}]: dispatch 2024-06-18T13:15:10.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:15:10.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-18T13:15:10.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.9", "id": [2, 3]}]': finished 2024-06-18T13:15:10.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.11", "id": [2, 3]}]': finished 2024-06-18T13:15:10.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.13", "id": [5, 3]}]': finished 2024-06-18T13:15:10.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "321.1e", "id": [5, 4]}]': finished 2024-06-18T13:15:10.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: osdmap e561: 8 total, 8 up, 8 in 2024-06-18T13:15:10.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-06-18T13:15:10.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-06-18T13:15:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:15:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:10 smithi179 ceph-mon[33312]: pgmap v855: 244 pgs: 244 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-06-18T13:15:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:10 smithi179 ceph-mon[33312]: osdmap e562: 8 total, 8 up, 8 in 2024-06-18T13:15:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:11.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:11.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:15:11.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:10 smithi167 ceph-mon[29496]: pgmap v855: 244 pgs: 244 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:11.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-06-18T13:15:11.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:10 smithi167 ceph-mon[29496]: osdmap e562: 8 total, 8 up, 8 in 2024-06-18T13:15:11.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:11.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:11.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:11.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:15:11.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:10 smithi167 ceph-mon[37701]: pgmap v855: 244 pgs: 244 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:11.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-06-18T13:15:11.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:10 smithi167 ceph-mon[37701]: osdmap e562: 8 total, 8 up, 8 in 2024-06-18T13:15:11.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:11.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-109"}]': finished 2024-06-18T13:15:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:12 smithi167 ceph-mon[29496]: osdmap e563: 8 total, 8 up, 8 in 2024-06-18T13:15:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:12 smithi167 ceph-mon[29496]: pgmap v858: 244 pgs: 244 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-109"}]': finished 2024-06-18T13:15:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:12 smithi167 ceph-mon[37701]: osdmap e563: 8 total, 8 up, 8 in 2024-06-18T13:15:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:12.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:12 smithi167 ceph-mon[37701]: pgmap v858: 244 pgs: 244 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:13.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-109"}]': finished 2024-06-18T13:15:13.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:12 smithi179 ceph-mon[33312]: osdmap e563: 8 total, 8 up, 8 in 2024-06-18T13:15:13.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:13.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]: dispatch 2024-06-18T13:15:13.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:12 smithi179 ceph-mon[33312]: pgmap v858: 244 pgs: 244 active+clean; 583 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]': finished 2024-06-18T13:15:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:13 smithi167 ceph-mon[37701]: osdmap e564: 8 total, 8 up, 8 in 2024-06-18T13:15:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]': finished 2024-06-18T13:15:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:13 smithi167 ceph-mon[29496]: osdmap e564: 8 total, 8 up, 8 in 2024-06-18T13:15:14.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-109", "tierpool": "test-rados-api-smithi167-74266-109-cache"}]': finished 2024-06-18T13:15:14.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:13 smithi179 ceph-mon[33312]: osdmap e564: 8 total, 8 up, 8 in 2024-06-18T13:15:14.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:14 smithi167 ceph-mon[37701]: osdmap e565: 8 total, 8 up, 8 in 2024-06-18T13:15:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:14 smithi167 ceph-mon[37701]: pgmap v861: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:14 smithi167 ceph-mon[29496]: osdmap e565: 8 total, 8 up, 8 in 2024-06-18T13:15:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:14 smithi167 ceph-mon[29496]: pgmap v861: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:15.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:14 smithi179 ceph-mon[33312]: osdmap e565: 8 total, 8 up, 8 in 2024-06-18T13:15:15.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:14 smithi179 ceph-mon[33312]: pgmap v861: 212 pgs: 212 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:15.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:15:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:15:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:15:16.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:15 smithi179 ceph-mon[33312]: osdmap e566: 8 total, 8 up, 8 in 2024-06-18T13:15:16.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:16.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:15 smithi167 ceph-mon[37701]: osdmap e566: 8 total, 8 up, 8 in 2024-06-18T13:15:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:16.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:15 smithi167 ceph-mon[29496]: osdmap e566: 8 total, 8 up, 8 in 2024-06-18T13:15:16.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:16.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:17.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]': finished 2024-06-18T13:15:17.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:16 smithi179 ceph-mon[33312]: osdmap e567: 8 total, 8 up, 8 in 2024-06-18T13:15:17.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:17.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:17.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:16 smithi179 ceph-mon[33312]: pgmap v864: 204 pgs: 204 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:17.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]': finished 2024-06-18T13:15:17.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]': finished 2024-06-18T13:15:17.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:16 smithi167 ceph-mon[37701]: osdmap e567: 8 total, 8 up, 8 in 2024-06-18T13:15:17.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:17.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:17.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:16 smithi167 ceph-mon[37701]: pgmap v864: 204 pgs: 204 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:17.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]': finished 2024-06-18T13:15:17.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-109"}]': finished 2024-06-18T13:15:17.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:16 smithi167 ceph-mon[29496]: osdmap e567: 8 total, 8 up, 8 in 2024-06-18T13:15:17.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3372379564' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:17.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]: dispatch 2024-06-18T13:15:17.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:16 smithi167 ceph-mon[29496]: pgmap v864: 204 pgs: 204 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:17.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-109"}]': finished 2024-06-18T13:15:18.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:17 smithi179 ceph-mon[33312]: osdmap e568: 8 total, 8 up, 8 in 2024-06-18T13:15:18.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:17 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:15:18.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:17 smithi167 ceph-mon[37701]: osdmap e568: 8 total, 8 up, 8 in 2024-06-18T13:15:18.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:17 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:15:18.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:17 smithi167 ceph-mon[29496]: osdmap e568: 8 total, 8 up, 8 in 2024-06-18T13:15:18.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:17 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:15:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:18 smithi179 ceph-mon[33312]: osdmap e569: 8 total, 8 up, 8 in 2024-06-18T13:15:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:15:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:18 smithi179 ceph-mon[33312]: pgmap v867: 196 pgs: 196 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:15:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104"}]': finished 2024-06-18T13:15:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:18 smithi179 ceph-mon[33312]: osdmap e570: 8 total, 8 up, 8 in 2024-06-18T13:15:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:15:19.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:18 smithi167 ceph-mon[37701]: osdmap e569: 8 total, 8 up, 8 in 2024-06-18T13:15:19.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:15:19.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:18 smithi167 ceph-mon[37701]: pgmap v867: 196 pgs: 196 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:15:19.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104"}]': finished 2024-06-18T13:15:19.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:18 smithi167 ceph-mon[37701]: osdmap e570: 8 total, 8 up, 8 in 2024-06-18T13:15:19.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:15:19.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:18 smithi167 ceph-mon[29496]: osdmap e569: 8 total, 8 up, 8 in 2024-06-18T13:15:19.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:15:19.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:18 smithi167 ceph-mon[29496]: pgmap v867: 196 pgs: 196 active+clean; 583 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:15:19.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi167-74266-104"}]': finished 2024-06-18T13:15:19.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:18 smithi167 ceph-mon[29496]: osdmap e570: 8 total, 8 up, 8 in 2024-06-18T13:15:19.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi167-74266-104"}]: dispatch 2024-06-18T13:15:19.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:15:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:15:20.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:19 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi167-74266-104"}]': finished 2024-06-18T13:15:20.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:19 smithi179 ceph-mon[33312]: osdmap e571: 8 total, 8 up, 8 in 2024-06-18T13:15:20.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:19 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:20.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi167-74266-104"}]': finished 2024-06-18T13:15:20.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:19 smithi167 ceph-mon[37701]: osdmap e571: 8 total, 8 up, 8 in 2024-06-18T13:15:20.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:19 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:20.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2283091615' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi167-74266-104"}]': finished 2024-06-18T13:15:20.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:19 smithi167 ceph-mon[29496]: osdmap e571: 8 total, 8 up, 8 in 2024-06-18T13:15:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:15:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:15:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:20 smithi179 ceph-mon[33312]: pgmap v870: 196 pgs: 196 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-18T13:15:21.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:21.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:21.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:21.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:21.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:15:21.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:15:21.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:20 smithi167 ceph-mon[37701]: pgmap v870: 196 pgs: 196 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-18T13:15:21.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:21.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:21.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:21.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:21.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:15:21.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-18T13:15:21.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:20 smithi167 ceph-mon[29496]: pgmap v870: 196 pgs: 196 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-18T13:15:22.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:15:22.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:22 smithi179 ceph-mon[33312]: osdmap e572: 8 total, 8 up, 8 in 2024-06-18T13:15:22.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:22.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:22.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:15:22.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:22 smithi167 ceph-mon[29496]: osdmap e572: 8 total, 8 up, 8 in 2024-06-18T13:15:22.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:22.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:22.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi167-74266-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-18T13:15:22.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:22 smithi167 ceph-mon[37701]: osdmap e572: 8 total, 8 up, 8 in 2024-06-18T13:15:22.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:22.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:23.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:23 smithi179 ceph-mon[33312]: osdmap e573: 8 total, 8 up, 8 in 2024-06-18T13:15:23.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:23 smithi179 ceph-mon[33312]: pgmap v873: 196 pgs: 196 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:23.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:15:23.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:23 smithi179 ceph-mon[33312]: osdmap e574: 8 total, 8 up, 8 in 2024-06-18T13:15:23.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:23 smithi167 ceph-mon[37701]: osdmap e573: 8 total, 8 up, 8 in 2024-06-18T13:15:23.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:23 smithi167 ceph-mon[37701]: pgmap v873: 196 pgs: 196 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:23.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:15:23.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:23 smithi167 ceph-mon[37701]: osdmap e574: 8 total, 8 up, 8 in 2024-06-18T13:15:23.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:23 smithi167 ceph-mon[29496]: osdmap e573: 8 total, 8 up, 8 in 2024-06-18T13:15:23.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:23 smithi167 ceph-mon[29496]: pgmap v873: 196 pgs: 196 active+clean; 583 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:23.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi167-74266-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:15:23.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:23 smithi167 ceph-mon[29496]: osdmap e574: 8 total, 8 up, 8 in 2024-06-18T13:15:24.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:24 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:15:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:24 smithi179 ceph-mon[33312]: osdmap e575: 8 total, 8 up, 8 in 2024-06-18T13:15:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:24.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:24 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:15:24.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:24 smithi167 ceph-mon[37701]: osdmap e575: 8 total, 8 up, 8 in 2024-06-18T13:15:24.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:24.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:24.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:24 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:15:24.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:24 smithi167 ceph-mon[29496]: osdmap e575: 8 total, 8 up, 8 in 2024-06-18T13:15:24.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:24.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:25.288 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:25 smithi179 ceph-mon[33312]: pgmap v876: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:25.288 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:15:25.288 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:25 smithi179 ceph-mon[33312]: osdmap e576: 8 total, 8 up, 8 in 2024-06-18T13:15:25.288 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:25.288 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:25.288 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:15:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:15:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:15:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:15:25.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:25 smithi167 ceph-mon[29496]: pgmap v876: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:25.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:15:25.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:25 smithi167 ceph-mon[29496]: osdmap e576: 8 total, 8 up, 8 in 2024-06-18T13:15:25.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:25.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:25.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:15:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:25 smithi167 ceph-mon[37701]: pgmap v876: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:15:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:25 smithi167 ceph-mon[37701]: osdmap e576: 8 total, 8 up, 8 in 2024-06-18T13:15:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:15:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:15:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:26 smithi179 ceph-mon[33312]: osdmap e577: 8 total, 8 up, 8 in 2024-06-18T13:15:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-112"}]: dispatch 2024-06-18T13:15:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-112"}]: dispatch 2024-06-18T13:15:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:26 smithi179 ceph-mon[33312]: pgmap v879: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:27.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:15:27.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:26 smithi167 ceph-mon[29496]: osdmap e577: 8 total, 8 up, 8 in 2024-06-18T13:15:27.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-112"}]: dispatch 2024-06-18T13:15:27.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-112"}]: dispatch 2024-06-18T13:15:27.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:26 smithi167 ceph-mon[29496]: pgmap v879: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:15:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:26 smithi167 ceph-mon[37701]: osdmap e577: 8 total, 8 up, 8 in 2024-06-18T13:15:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-112"}]: dispatch 2024-06-18T13:15:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-112"}]: dispatch 2024-06-18T13:15:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:26 smithi167 ceph-mon[37701]: pgmap v879: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:28.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-112"}]': finished 2024-06-18T13:15:28.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:27 smithi179 ceph-mon[33312]: osdmap e578: 8 total, 8 up, 8 in 2024-06-18T13:15:28.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:28.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:28.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-112"}]': finished 2024-06-18T13:15:28.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:27 smithi167 ceph-mon[37701]: osdmap e578: 8 total, 8 up, 8 in 2024-06-18T13:15:28.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:28.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:28.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-112"}]': finished 2024-06-18T13:15:28.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:27 smithi167 ceph-mon[29496]: osdmap e578: 8 total, 8 up, 8 in 2024-06-18T13:15:28.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:28.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:29.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:15:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:28 smithi179 ceph-mon[33312]: osdmap e579: 8 total, 8 up, 8 in 2024-06-18T13:15:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112"}]: dispatch 2024-06-18T13:15:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112"}]: dispatch 2024-06-18T13:15:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:28 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:15:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:28 smithi179 ceph-mon[33312]: pgmap v882: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:15:29.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:15:29.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:28 smithi167 ceph-mon[37701]: osdmap e579: 8 total, 8 up, 8 in 2024-06-18T13:15:29.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112"}]: dispatch 2024-06-18T13:15:29.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112"}]: dispatch 2024-06-18T13:15:29.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:28 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:15:29.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:28 smithi167 ceph-mon[37701]: pgmap v882: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:15:29.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:15:29.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:28 smithi167 ceph-mon[29496]: osdmap e579: 8 total, 8 up, 8 in 2024-06-18T13:15:29.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112"}]: dispatch 2024-06-18T13:15:29.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112"}]: dispatch 2024-06-18T13:15:29.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:28 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:15:29.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:28 smithi167 ceph-mon[29496]: pgmap v882: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:15:29.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:15:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:15:30.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112"}]': finished 2024-06-18T13:15:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:29 smithi179 ceph-mon[33312]: osdmap e580: 8 total, 8 up, 8 in 2024-06-18T13:15:30.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:29 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:30.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112"}]': finished 2024-06-18T13:15:30.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:29 smithi167 ceph-mon[37701]: osdmap e580: 8 total, 8 up, 8 in 2024-06-18T13:15:30.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:29 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:30.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-112"}]': finished 2024-06-18T13:15:30.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:29 smithi167 ceph-mon[29496]: osdmap e580: 8 total, 8 up, 8 in 2024-06-18T13:15:30.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:29 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:31.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:30 smithi179 ceph-mon[33312]: osdmap e581: 8 total, 8 up, 8 in 2024-06-18T13:15:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:30 smithi179 ceph-mon[33312]: pgmap v885: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:31.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:30 smithi167 ceph-mon[29496]: osdmap e581: 8 total, 8 up, 8 in 2024-06-18T13:15:31.226 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:30 smithi167 ceph-mon[29496]: pgmap v885: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:31.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:30 smithi167 ceph-mon[37701]: osdmap e581: 8 total, 8 up, 8 in 2024-06-18T13:15:31.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:30 smithi167 ceph-mon[37701]: pgmap v885: 204 pgs: 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:32.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:31 smithi179 ceph-mon[33312]: osdmap e582: 8 total, 8 up, 8 in 2024-06-18T13:15:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:32.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:31 smithi167 ceph-mon[29496]: osdmap e582: 8 total, 8 up, 8 in 2024-06-18T13:15:32.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:32.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:32.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:31 smithi167 ceph-mon[37701]: osdmap e582: 8 total, 8 up, 8 in 2024-06-18T13:15:32.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:32.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:15:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:32 smithi179 ceph-mon[33312]: osdmap e583: 8 total, 8 up, 8 in 2024-06-18T13:15:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:32 smithi179 ceph-mon[33312]: pgmap v888: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:33.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:15:33.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:32 smithi167 ceph-mon[37701]: osdmap e583: 8 total, 8 up, 8 in 2024-06-18T13:15:33.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:33.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:33.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:32 smithi167 ceph-mon[37701]: pgmap v888: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:33.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:15:33.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:32 smithi167 ceph-mon[29496]: osdmap e583: 8 total, 8 up, 8 in 2024-06-18T13:15:33.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:33.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:33.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:32 smithi167 ceph-mon[29496]: pgmap v888: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:34.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:33 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:15:34.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:33 smithi179 ceph-mon[33312]: osdmap e584: 8 total, 8 up, 8 in 2024-06-18T13:15:34.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-114"}]: dispatch 2024-06-18T13:15:34.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:33 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-114"}]: dispatch 2024-06-18T13:15:34.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:33 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:15:34.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:33 smithi167 ceph-mon[37701]: osdmap e584: 8 total, 8 up, 8 in 2024-06-18T13:15:34.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-114"}]: dispatch 2024-06-18T13:15:34.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:33 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-114"}]: dispatch 2024-06-18T13:15:34.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:33 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:15:34.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:33 smithi167 ceph-mon[29496]: osdmap e584: 8 total, 8 up, 8 in 2024-06-18T13:15:34.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-114"}]: dispatch 2024-06-18T13:15:34.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:33 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-114"}]: dispatch 2024-06-18T13:15:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-114"}]': finished 2024-06-18T13:15:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:34 smithi179 ceph-mon[33312]: osdmap e585: 8 total, 8 up, 8 in 2024-06-18T13:15:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-114", "mode": "writeback"}]: dispatch 2024-06-18T13:15:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-114", "mode": "writeback"}]: dispatch 2024-06-18T13:15:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:34 smithi179 ceph-mon[33312]: pgmap v891: 236 pgs: 236 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:35.226 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:15:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:15:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:15:35.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-114"}]': finished 2024-06-18T13:15:35.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:34 smithi167 ceph-mon[29496]: osdmap e585: 8 total, 8 up, 8 in 2024-06-18T13:15:35.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-114", "mode": "writeback"}]: dispatch 2024-06-18T13:15:35.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-114", "mode": "writeback"}]: dispatch 2024-06-18T13:15:35.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:34 smithi167 ceph-mon[29496]: pgmap v891: 236 pgs: 236 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:35.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-114"}]': finished 2024-06-18T13:15:35.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:34 smithi167 ceph-mon[37701]: osdmap e585: 8 total, 8 up, 8 in 2024-06-18T13:15:35.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-114", "mode": "writeback"}]: dispatch 2024-06-18T13:15:35.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-114", "mode": "writeback"}]: dispatch 2024-06-18T13:15:35.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:34 smithi167 ceph-mon[37701]: pgmap v891: 236 pgs: 236 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:35 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:15:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-114", "mode": "writeback"}]': finished 2024-06-18T13:15:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:35 smithi179 ceph-mon[33312]: osdmap e586: 8 total, 8 up, 8 in 2024-06-18T13:15:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:36.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:36.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:35 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:15:36.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-114", "mode": "writeback"}]': finished 2024-06-18T13:15:36.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:35 smithi167 ceph-mon[29496]: osdmap e586: 8 total, 8 up, 8 in 2024-06-18T13:15:36.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:36.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:36.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:35 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:15:36.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-114", "mode": "writeback"}]': finished 2024-06-18T13:15:36.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:35 smithi167 ceph-mon[37701]: osdmap e586: 8 total, 8 up, 8 in 2024-06-18T13:15:36.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:36.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:15:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:15:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:36 smithi179 ceph-mon[33312]: osdmap e587: 8 total, 8 up, 8 in 2024-06-18T13:15:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114"}]: dispatch 2024-06-18T13:15:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114"}]: dispatch 2024-06-18T13:15:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:36 smithi179 ceph-mon[33312]: pgmap v894: 236 pgs: 236 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:37.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:15:37.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:36 smithi167 ceph-mon[29496]: osdmap e587: 8 total, 8 up, 8 in 2024-06-18T13:15:37.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114"}]: dispatch 2024-06-18T13:15:37.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114"}]: dispatch 2024-06-18T13:15:37.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:36 smithi167 ceph-mon[29496]: pgmap v894: 236 pgs: 236 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:37.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:15:37.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:36 smithi167 ceph-mon[37701]: osdmap e587: 8 total, 8 up, 8 in 2024-06-18T13:15:37.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114"}]: dispatch 2024-06-18T13:15:37.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114"}]: dispatch 2024-06-18T13:15:37.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:36 smithi167 ceph-mon[37701]: pgmap v894: 236 pgs: 236 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:15:38.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:38 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:15:38.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114"}]': finished 2024-06-18T13:15:38.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:38 smithi179 ceph-mon[33312]: osdmap e588: 8 total, 8 up, 8 in 2024-06-18T13:15:38.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:38 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:15:38.476 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114"}]': finished 2024-06-18T13:15:38.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:38 smithi167 ceph-mon[29496]: osdmap e588: 8 total, 8 up, 8 in 2024-06-18T13:15:38.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:38 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:15:38.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-114"}]': finished 2024-06-18T13:15:38.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:38 smithi167 ceph-mon[37701]: osdmap e588: 8 total, 8 up, 8 in 2024-06-18T13:15:39.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:15:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:15:39.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:39 smithi179 ceph-mon[33312]: osdmap e589: 8 total, 8 up, 8 in 2024-06-18T13:15:39.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:39 smithi179 ceph-mon[33312]: pgmap v897: 204 pgs: 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:15:39.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:39 smithi179 ceph-mon[33312]: osdmap e590: 8 total, 8 up, 8 in 2024-06-18T13:15:39.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:39.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:39 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:39.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:39 smithi167 ceph-mon[29496]: osdmap e589: 8 total, 8 up, 8 in 2024-06-18T13:15:39.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:39 smithi167 ceph-mon[29496]: pgmap v897: 204 pgs: 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:15:39.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:39 smithi167 ceph-mon[29496]: osdmap e590: 8 total, 8 up, 8 in 2024-06-18T13:15:39.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:39.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:39 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:39.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:39 smithi167 ceph-mon[37701]: osdmap e589: 8 total, 8 up, 8 in 2024-06-18T13:15:39.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:39 smithi167 ceph-mon[37701]: pgmap v897: 204 pgs: 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:15:39.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:39 smithi167 ceph-mon[37701]: osdmap e590: 8 total, 8 up, 8 in 2024-06-18T13:15:39.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:39.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:39 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:15:40.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:40.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:15:40.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:40 smithi167 ceph-mon[29496]: osdmap e591: 8 total, 8 up, 8 in 2024-06-18T13:15:40.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:15:40.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:40.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:15:40.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:40 smithi167 ceph-mon[37701]: osdmap e591: 8 total, 8 up, 8 in 2024-06-18T13:15:40.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:15:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:15:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:40 smithi179 ceph-mon[33312]: osdmap e591: 8 total, 8 up, 8 in 2024-06-18T13:15:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:15:41.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:41 smithi167 ceph-mon[37701]: pgmap v900: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 675 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:41.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:41 smithi167 ceph-mon[37701]: osdmap e592: 8 total, 8 up, 8 in 2024-06-18T13:15:41.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:41.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:41.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:41 smithi167 ceph-mon[29496]: pgmap v900: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 675 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:41.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:41 smithi167 ceph-mon[29496]: osdmap e592: 8 total, 8 up, 8 in 2024-06-18T13:15:41.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:41.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:41.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:41 smithi179 ceph-mon[33312]: pgmap v900: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 675 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:41.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:41 smithi179 ceph-mon[33312]: osdmap e592: 8 total, 8 up, 8 in 2024-06-18T13:15:41.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:41.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:15:43.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:15:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:42 smithi179 ceph-mon[33312]: osdmap e593: 8 total, 8 up, 8 in 2024-06-18T13:15:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-116"}]: dispatch 2024-06-18T13:15:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-116"}]: dispatch 2024-06-18T13:15:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:42 smithi179 ceph-mon[33312]: pgmap v903: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 675 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:42 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:15:43.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:15:43.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:42 smithi167 ceph-mon[37701]: osdmap e593: 8 total, 8 up, 8 in 2024-06-18T13:15:43.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-116"}]: dispatch 2024-06-18T13:15:43.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-116"}]: dispatch 2024-06-18T13:15:43.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:42 smithi167 ceph-mon[37701]: pgmap v903: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 675 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:43.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:42 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:15:43.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:15:43.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:42 smithi167 ceph-mon[29496]: osdmap e593: 8 total, 8 up, 8 in 2024-06-18T13:15:43.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-116"}]: dispatch 2024-06-18T13:15:43.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-116"}]: dispatch 2024-06-18T13:15:43.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:42 smithi167 ceph-mon[29496]: pgmap v903: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 675 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:43.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:42 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:15:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-116"}]': finished 2024-06-18T13:15:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:43 smithi179 ceph-mon[33312]: osdmap e594: 8 total, 8 up, 8 in 2024-06-18T13:15:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-116", "mode": "writeback"}]: dispatch 2024-06-18T13:15:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-116", "mode": "writeback"}]: dispatch 2024-06-18T13:15:44.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-116"}]': finished 2024-06-18T13:15:44.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:43 smithi167 ceph-mon[37701]: osdmap e594: 8 total, 8 up, 8 in 2024-06-18T13:15:44.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-116", "mode": "writeback"}]: dispatch 2024-06-18T13:15:44.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-116", "mode": "writeback"}]: dispatch 2024-06-18T13:15:44.184 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-116"}]': finished 2024-06-18T13:15:44.184 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:43 smithi167 ceph-mon[29496]: osdmap e594: 8 total, 8 up, 8 in 2024-06-18T13:15:44.184 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-116", "mode": "writeback"}]: dispatch 2024-06-18T13:15:44.184 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-116", "mode": "writeback"}]: dispatch 2024-06-18T13:15:45.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:44 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:15:45.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:44 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-116", "mode": "writeback"}]': finished 2024-06-18T13:15:45.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:44 smithi167 ceph-mon[37701]: osdmap e595: 8 total, 8 up, 8 in 2024-06-18T13:15:45.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.7"}]: dispatch 2024-06-18T13:15:45.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:44 smithi167 ceph-mon[37701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.7"}]: dispatch 2024-06-18T13:15:45.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:44 smithi167 ceph-mon[37701]: pgmap v906: 236 pgs: 236 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-18T13:15:45.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:44 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:15:45.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:44 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-116", "mode": "writeback"}]': finished 2024-06-18T13:15:45.032 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:44 smithi167 ceph-mon[29496]: osdmap e595: 8 total, 8 up, 8 in 2024-06-18T13:15:45.033 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.7"}]: dispatch 2024-06-18T13:15:45.033 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:44 smithi167 ceph-mon[29496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.7"}]: dispatch 2024-06-18T13:15:45.033 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:44 smithi167 ceph-mon[29496]: pgmap v906: 236 pgs: 236 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-18T13:15:45.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:44 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:15:45.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:44 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-116", "mode": "writeback"}]': finished 2024-06-18T13:15:45.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:44 smithi179 ceph-mon[33312]: osdmap e595: 8 total, 8 up, 8 in 2024-06-18T13:15:45.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.7"}]: dispatch 2024-06-18T13:15:45.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:44 smithi179 ceph-mon[33312]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.7"}]: dispatch 2024-06-18T13:15:45.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:44 smithi179 ceph-mon[33312]: pgmap v906: 236 pgs: 236 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-18T13:15:45.313 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:15:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:15:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:15:46.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:46 smithi179 ceph-mon[33312]: 325.7 scrub starts 2024-06-18T13:15:46.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:46 smithi179 ceph-mon[33312]: 325.7 scrub ok 2024-06-18T13:15:46.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:46 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:46.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:46 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:46 smithi167 ceph-mon[29496]: 325.7 scrub starts 2024-06-18T13:15:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:46 smithi167 ceph-mon[29496]: 325.7 scrub ok 2024-06-18T13:15:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:46 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:46 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:46 smithi167 ceph-mon[37701]: 325.7 scrub starts 2024-06-18T13:15:46.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:46 smithi167 ceph-mon[37701]: 325.7 scrub ok 2024-06-18T13:15:46.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:46 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:46.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:46 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:47 smithi167 ceph-mon[29496]: pgmap v907: 236 pgs: 236 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 943 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-06-18T13:15:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:47 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:47 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:47 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:15:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:47 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:15:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:47 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:47 smithi167 ceph-mon[37701]: pgmap v907: 236 pgs: 236 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 943 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-06-18T13:15:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:47 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:47 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:47 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:15:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:47 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:15:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:47 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:47.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:47 smithi179 ceph-mon[33312]: pgmap v907: 236 pgs: 236 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 943 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-06-18T13:15:47.898 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:47 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:47.898 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:47 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:47.898 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:47 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:15:47.898 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:47 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:15:47.898 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:47 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:49.437 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:15:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:15:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:49 smithi167 ceph-mon[37701]: pgmap v908: 236 pgs: 236 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 797 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-06-18T13:15:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:49 smithi167 ceph-mon[29496]: pgmap v908: 236 pgs: 236 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 797 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-06-18T13:15:49.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:49 smithi179 ceph-mon[33312]: pgmap v908: 236 pgs: 236 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 797 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-06-18T13:15:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:50.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:15:51.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:51 smithi167 ceph-mon[37701]: pgmap v909: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-06-18T13:15:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:51 smithi167 ceph-mon[29496]: pgmap v909: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-06-18T13:15:51.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:51 smithi179 ceph-mon[33312]: pgmap v909: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-06-18T13:15:53.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:53 smithi167 ceph-mon[37701]: pgmap v910: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 761 B/s wr, 2 op/s 2024-06-18T13:15:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:53 smithi167 ceph-mon[29496]: pgmap v910: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 761 B/s wr, 2 op/s 2024-06-18T13:15:53.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:53 smithi179 ceph-mon[33312]: pgmap v910: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 761 B/s wr, 2 op/s 2024-06-18T13:15:55.450 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:15:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:15:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:15:55.712 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:55 smithi179 ceph-mon[33312]: pgmap v911: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:55.712 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:55.712 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:15:55.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:55 smithi167 ceph-mon[37701]: pgmap v911: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:15:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:55 smithi167 ceph-mon[29496]: pgmap v911: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:15:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:15:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:15:57.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:57 smithi167 ceph-mon[29496]: pgmap v912: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-06-18T13:15:57.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:57 smithi167 ceph-mon[37701]: pgmap v912: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-06-18T13:15:57.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:57 smithi179 ceph-mon[33312]: pgmap v912: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-06-18T13:15:59.471 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:15:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:15:59.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:15:59 smithi167 ceph-mon[29496]: pgmap v913: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-06-18T13:15:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:15:59 smithi167 ceph-mon[37701]: pgmap v913: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-06-18T13:15:59.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:15:59 smithi179 ceph-mon[33312]: pgmap v913: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-06-18T13:16:00.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:00 smithi179 ceph-mon[33312]: osdmap e596: 8 total, 8 up, 8 in 2024-06-18T13:16:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:00 smithi167 ceph-mon[37701]: osdmap e596: 8 total, 8 up, 8 in 2024-06-18T13:16:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:00 smithi167 ceph-mon[29496]: osdmap e596: 8 total, 8 up, 8 in 2024-06-18T13:16:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:01 smithi179 ceph-mon[33312]: pgmap v915: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:16:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:01 smithi179 ceph-mon[33312]: osdmap e597: 8 total, 8 up, 8 in 2024-06-18T13:16:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116"}]: dispatch 2024-06-18T13:16:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116"}]: dispatch 2024-06-18T13:16:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:01 smithi167 ceph-mon[37701]: pgmap v915: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:16:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:01 smithi167 ceph-mon[37701]: osdmap e597: 8 total, 8 up, 8 in 2024-06-18T13:16:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116"}]: dispatch 2024-06-18T13:16:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116"}]: dispatch 2024-06-18T13:16:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:01 smithi167 ceph-mon[29496]: pgmap v915: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:16:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:01 smithi167 ceph-mon[29496]: osdmap e597: 8 total, 8 up, 8 in 2024-06-18T13:16:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116"}]: dispatch 2024-06-18T13:16:01.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116"}]: dispatch 2024-06-18T13:16:02.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:02 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116"}]': finished 2024-06-18T13:16:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:02 smithi179 ceph-mon[33312]: osdmap e598: 8 total, 8 up, 8 in 2024-06-18T13:16:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:02 smithi179 ceph-mon[33312]: pgmap v918: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:16:02.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:02 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116"}]': finished 2024-06-18T13:16:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:02 smithi167 ceph-mon[37701]: osdmap e598: 8 total, 8 up, 8 in 2024-06-18T13:16:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:02 smithi167 ceph-mon[37701]: pgmap v918: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:16:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:02 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-116"}]': finished 2024-06-18T13:16:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:02 smithi167 ceph-mon[29496]: osdmap e598: 8 total, 8 up, 8 in 2024-06-18T13:16:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:02 smithi167 ceph-mon[29496]: pgmap v918: 236 pgs: 236 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:16:03.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:03 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:03 smithi179 ceph-mon[33312]: osdmap e599: 8 total, 8 up, 8 in 2024-06-18T13:16:03.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:03 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:03 smithi167 ceph-mon[37701]: osdmap e599: 8 total, 8 up, 8 in 2024-06-18T13:16:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:03 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:03 smithi167 ceph-mon[29496]: osdmap e599: 8 total, 8 up, 8 in 2024-06-18T13:16:04.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:04 smithi179 ceph-mon[33312]: osdmap e600: 8 total, 8 up, 8 in 2024-06-18T13:16:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:04 smithi179 ceph-mon[33312]: pgmap v921: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-06-18T13:16:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:04 smithi167 ceph-mon[37701]: osdmap e600: 8 total, 8 up, 8 in 2024-06-18T13:16:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:04 smithi167 ceph-mon[37701]: pgmap v921: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-06-18T13:16:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:04 smithi167 ceph-mon[29496]: osdmap e600: 8 total, 8 up, 8 in 2024-06-18T13:16:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:04 smithi167 ceph-mon[29496]: pgmap v921: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-06-18T13:16:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:16:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:16:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:16:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:05 smithi179 ceph-mon[33312]: osdmap e601: 8 total, 8 up, 8 in 2024-06-18T13:16:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:05 smithi167 ceph-mon[37701]: osdmap e601: 8 total, 8 up, 8 in 2024-06-18T13:16:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:05.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:05 smithi167 ceph-mon[29496]: osdmap e601: 8 total, 8 up, 8 in 2024-06-18T13:16:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:06 smithi179 ceph-mon[33312]: osdmap e602: 8 total, 8 up, 8 in 2024-06-18T13:16:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:06 smithi179 ceph-mon[33312]: pgmap v924: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-06-18T13:16:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:06 smithi167 ceph-mon[37701]: osdmap e602: 8 total, 8 up, 8 in 2024-06-18T13:16:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:06 smithi167 ceph-mon[37701]: pgmap v924: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-06-18T13:16:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:06 smithi167 ceph-mon[29496]: osdmap e602: 8 total, 8 up, 8 in 2024-06-18T13:16:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:06 smithi167 ceph-mon[29496]: pgmap v924: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-06-18T13:16:07.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:07 smithi179 ceph-mon[33312]: osdmap e603: 8 total, 8 up, 8 in 2024-06-18T13:16:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-118"}]: dispatch 2024-06-18T13:16:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-118"}]: dispatch 2024-06-18T13:16:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:07 smithi167 ceph-mon[37701]: osdmap e603: 8 total, 8 up, 8 in 2024-06-18T13:16:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-118"}]: dispatch 2024-06-18T13:16:07.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-118"}]: dispatch 2024-06-18T13:16:07.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:07.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:07 smithi167 ceph-mon[29496]: osdmap e603: 8 total, 8 up, 8 in 2024-06-18T13:16:07.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-118"}]: dispatch 2024-06-18T13:16:07.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-118"}]: dispatch 2024-06-18T13:16:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-118"}]': finished 2024-06-18T13:16:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:08 smithi179 ceph-mon[33312]: osdmap e604: 8 total, 8 up, 8 in 2024-06-18T13:16:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-118", "mode": "writeback"}]: dispatch 2024-06-18T13:16:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-118", "mode": "writeback"}]: dispatch 2024-06-18T13:16:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:08 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:08 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-118", "mode": "writeback"}]': finished 2024-06-18T13:16:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:08 smithi179 ceph-mon[33312]: osdmap e605: 8 total, 8 up, 8 in 2024-06-18T13:16:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:08 smithi179 ceph-mon[33312]: pgmap v928: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-118"}]': finished 2024-06-18T13:16:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[37701]: osdmap e604: 8 total, 8 up, 8 in 2024-06-18T13:16:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-118", "mode": "writeback"}]: dispatch 2024-06-18T13:16:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-118", "mode": "writeback"}]: dispatch 2024-06-18T13:16:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-118", "mode": "writeback"}]': finished 2024-06-18T13:16:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[37701]: osdmap e605: 8 total, 8 up, 8 in 2024-06-18T13:16:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[37701]: pgmap v928: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-118"}]': finished 2024-06-18T13:16:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[29496]: osdmap e604: 8 total, 8 up, 8 in 2024-06-18T13:16:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-118", "mode": "writeback"}]: dispatch 2024-06-18T13:16:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-118", "mode": "writeback"}]: dispatch 2024-06-18T13:16:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-118", "mode": "writeback"}]': finished 2024-06-18T13:16:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[29496]: osdmap e605: 8 total, 8 up, 8 in 2024-06-18T13:16:08.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:08 smithi167 ceph-mon[29496]: pgmap v928: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:09.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:16:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:16:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:10 smithi179 ceph-mon[33312]: osdmap e606: 8 total, 8 up, 8 in 2024-06-18T13:16:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:16:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:10 smithi167 ceph-mon[29496]: osdmap e606: 8 total, 8 up, 8 in 2024-06-18T13:16:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:16:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:10 smithi167 ceph-mon[37701]: osdmap e606: 8 total, 8 up, 8 in 2024-06-18T13:16:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:10.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:10.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:16:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:11 smithi179 ceph-mon[33312]: osdmap e607: 8 total, 8 up, 8 in 2024-06-18T13:16:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118"}]: dispatch 2024-06-18T13:16:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118"}]: dispatch 2024-06-18T13:16:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:11 smithi179 ceph-mon[33312]: pgmap v931: 236 pgs: 236 active+clean; 583 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 541 B/s wr, 1 op/s 2024-06-18T13:16:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:11 smithi167 ceph-mon[29496]: osdmap e607: 8 total, 8 up, 8 in 2024-06-18T13:16:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118"}]: dispatch 2024-06-18T13:16:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118"}]: dispatch 2024-06-18T13:16:11.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:11 smithi167 ceph-mon[29496]: pgmap v931: 236 pgs: 236 active+clean; 583 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 541 B/s wr, 1 op/s 2024-06-18T13:16:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:11 smithi167 ceph-mon[37701]: osdmap e607: 8 total, 8 up, 8 in 2024-06-18T13:16:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118"}]: dispatch 2024-06-18T13:16:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118"}]: dispatch 2024-06-18T13:16:11.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:11 smithi167 ceph-mon[37701]: pgmap v931: 236 pgs: 236 active+clean; 583 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 541 B/s wr, 1 op/s 2024-06-18T13:16:12.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:12 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:12 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118"}]': finished 2024-06-18T13:16:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:12 smithi179 ceph-mon[33312]: osdmap e608: 8 total, 8 up, 8 in 2024-06-18T13:16:12.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:12 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:12.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:12 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118"}]': finished 2024-06-18T13:16:12.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:12 smithi167 ceph-mon[37701]: osdmap e608: 8 total, 8 up, 8 in 2024-06-18T13:16:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:12 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:12 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-118"}]': finished 2024-06-18T13:16:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:12 smithi167 ceph-mon[29496]: osdmap e608: 8 total, 8 up, 8 in 2024-06-18T13:16:13.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:13 smithi179 ceph-mon[33312]: osdmap e609: 8 total, 8 up, 8 in 2024-06-18T13:16:13.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:13 smithi179 ceph-mon[33312]: pgmap v934: 204 pgs: 204 active+clean; 583 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:16:13.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:13 smithi167 ceph-mon[37701]: osdmap e609: 8 total, 8 up, 8 in 2024-06-18T13:16:13.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:13 smithi167 ceph-mon[37701]: pgmap v934: 204 pgs: 204 active+clean; 583 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:16:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:13 smithi167 ceph-mon[29496]: osdmap e609: 8 total, 8 up, 8 in 2024-06-18T13:16:13.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:13 smithi167 ceph-mon[29496]: pgmap v934: 204 pgs: 204 active+clean; 583 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:16:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:14 smithi179 ceph-mon[33312]: osdmap e610: 8 total, 8 up, 8 in 2024-06-18T13:16:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:14 smithi167 ceph-mon[37701]: osdmap e610: 8 total, 8 up, 8 in 2024-06-18T13:16:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:14 smithi167 ceph-mon[29496]: osdmap e610: 8 total, 8 up, 8 in 2024-06-18T13:16:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:16:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:16:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:16:15.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:15.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:15 smithi167 ceph-mon[29496]: osdmap e611: 8 total, 8 up, 8 in 2024-06-18T13:16:15.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:15.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:15.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:15 smithi167 ceph-mon[29496]: pgmap v937: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:15.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:15.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:15 smithi167 ceph-mon[37701]: osdmap e611: 8 total, 8 up, 8 in 2024-06-18T13:16:15.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:15.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:15.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:15 smithi167 ceph-mon[37701]: pgmap v937: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:15.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:15.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:15 smithi179 ceph-mon[33312]: osdmap e611: 8 total, 8 up, 8 in 2024-06-18T13:16:15.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:15.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:15.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:15 smithi179 ceph-mon[33312]: pgmap v937: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:16.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:16 smithi179 ceph-mon[33312]: osdmap e612: 8 total, 8 up, 8 in 2024-06-18T13:16:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-120"}]: dispatch 2024-06-18T13:16:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-120"}]: dispatch 2024-06-18T13:16:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:16 smithi167 ceph-mon[29496]: osdmap e612: 8 total, 8 up, 8 in 2024-06-18T13:16:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-120"}]: dispatch 2024-06-18T13:16:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-120"}]: dispatch 2024-06-18T13:16:16.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:16.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:16 smithi167 ceph-mon[37701]: osdmap e612: 8 total, 8 up, 8 in 2024-06-18T13:16:16.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-120"}]: dispatch 2024-06-18T13:16:16.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-120"}]: dispatch 2024-06-18T13:16:17.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-120"}]': finished 2024-06-18T13:16:17.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:17 smithi179 ceph-mon[33312]: osdmap e613: 8 total, 8 up, 8 in 2024-06-18T13:16:17.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-120", "mode": "writeback"}]: dispatch 2024-06-18T13:16:17.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-120", "mode": "writeback"}]: dispatch 2024-06-18T13:16:17.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:17 smithi179 ceph-mon[33312]: pgmap v940: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-120"}]': finished 2024-06-18T13:16:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:17 smithi167 ceph-mon[37701]: osdmap e613: 8 total, 8 up, 8 in 2024-06-18T13:16:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-120", "mode": "writeback"}]: dispatch 2024-06-18T13:16:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-120", "mode": "writeback"}]: dispatch 2024-06-18T13:16:17.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:17 smithi167 ceph-mon[37701]: pgmap v940: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:17.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-120"}]': finished 2024-06-18T13:16:17.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:17 smithi167 ceph-mon[29496]: osdmap e613: 8 total, 8 up, 8 in 2024-06-18T13:16:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-120", "mode": "writeback"}]: dispatch 2024-06-18T13:16:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-120", "mode": "writeback"}]: dispatch 2024-06-18T13:16:17.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:17 smithi167 ceph-mon[29496]: pgmap v940: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:18.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:18 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-120", "mode": "writeback"}]': finished 2024-06-18T13:16:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:18 smithi179 ceph-mon[33312]: osdmap e614: 8 total, 8 up, 8 in 2024-06-18T13:16:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:18 smithi179 ceph-mon[33312]: osdmap e615: 8 total, 8 up, 8 in 2024-06-18T13:16:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120"}]: dispatch 2024-06-18T13:16:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120"}]: dispatch 2024-06-18T13:16:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-120", "mode": "writeback"}]': finished 2024-06-18T13:16:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[37701]: osdmap e614: 8 total, 8 up, 8 in 2024-06-18T13:16:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[37701]: osdmap e615: 8 total, 8 up, 8 in 2024-06-18T13:16:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120"}]: dispatch 2024-06-18T13:16:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120"}]: dispatch 2024-06-18T13:16:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-120", "mode": "writeback"}]': finished 2024-06-18T13:16:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[29496]: osdmap e614: 8 total, 8 up, 8 in 2024-06-18T13:16:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[29496]: osdmap e615: 8 total, 8 up, 8 in 2024-06-18T13:16:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120"}]: dispatch 2024-06-18T13:16:18.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120"}]: dispatch 2024-06-18T13:16:19.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:16:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:16:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:19 smithi179 ceph-mon[33312]: pgmap v943: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:19 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120"}]': finished 2024-06-18T13:16:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:19 smithi179 ceph-mon[33312]: osdmap e616: 8 total, 8 up, 8 in 2024-06-18T13:16:19.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:19 smithi167 ceph-mon[37701]: pgmap v943: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:19 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120"}]': finished 2024-06-18T13:16:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:19 smithi167 ceph-mon[37701]: osdmap e616: 8 total, 8 up, 8 in 2024-06-18T13:16:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:19 smithi167 ceph-mon[29496]: pgmap v943: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:19 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-120"}]': finished 2024-06-18T13:16:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:19 smithi167 ceph-mon[29496]: osdmap e616: 8 total, 8 up, 8 in 2024-06-18T13:16:20.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:20 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:20 smithi167 ceph-mon[37701]: osdmap e617: 8 total, 8 up, 8 in 2024-06-18T13:16:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:20 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:20 smithi167 ceph-mon[29496]: osdmap e617: 8 total, 8 up, 8 in 2024-06-18T13:16:20.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:20 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:20 smithi179 ceph-mon[33312]: osdmap e617: 8 total, 8 up, 8 in 2024-06-18T13:16:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:21 smithi167 ceph-mon[37701]: pgmap v946: 204 pgs: 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:16:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:21 smithi167 ceph-mon[37701]: osdmap e618: 8 total, 8 up, 8 in 2024-06-18T13:16:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:21.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:21 smithi167 ceph-mon[29496]: pgmap v946: 204 pgs: 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:16:21.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:21 smithi167 ceph-mon[29496]: osdmap e618: 8 total, 8 up, 8 in 2024-06-18T13:16:21.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:21.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:21.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:21 smithi179 ceph-mon[33312]: pgmap v946: 204 pgs: 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:16:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:21 smithi179 ceph-mon[33312]: osdmap e618: 8 total, 8 up, 8 in 2024-06-18T13:16:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:23.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:23 smithi179 ceph-mon[33312]: osdmap e619: 8 total, 8 up, 8 in 2024-06-18T13:16:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:23.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:23 smithi179 ceph-mon[33312]: pgmap v949: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:16:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:23.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:23 smithi167 ceph-mon[37701]: osdmap e619: 8 total, 8 up, 8 in 2024-06-18T13:16:23.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:23.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:23.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:23 smithi167 ceph-mon[37701]: pgmap v949: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:16:23.785 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:23.785 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:23 smithi167 ceph-mon[29496]: osdmap e619: 8 total, 8 up, 8 in 2024-06-18T13:16:23.785 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:23.785 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:23.785 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:23 smithi167 ceph-mon[29496]: pgmap v949: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:16:24.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:24.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:24 smithi179 ceph-mon[33312]: osdmap e620: 8 total, 8 up, 8 in 2024-06-18T13:16:24.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-122"}]: dispatch 2024-06-18T13:16:24.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-122"}]: dispatch 2024-06-18T13:16:24.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:24 smithi167 ceph-mon[29496]: osdmap e620: 8 total, 8 up, 8 in 2024-06-18T13:16:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-122"}]: dispatch 2024-06-18T13:16:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-122"}]: dispatch 2024-06-18T13:16:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:24 smithi167 ceph-mon[37701]: osdmap e620: 8 total, 8 up, 8 in 2024-06-18T13:16:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-122"}]: dispatch 2024-06-18T13:16:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-122"}]: dispatch 2024-06-18T13:16:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:16:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:16:25.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-122"}]': finished 2024-06-18T13:16:25.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-mon[29496]: osdmap e621: 8 total, 8 up, 8 in 2024-06-18T13:16:25.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-122", "mode": "writeback"}]: dispatch 2024-06-18T13:16:25.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-122", "mode": "writeback"}]: dispatch 2024-06-18T13:16:25.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-mon[29496]: pgmap v952: 236 pgs: 236 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:16:25.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:16:25.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:16:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-122"}]': finished 2024-06-18T13:16:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-mon[37701]: osdmap e621: 8 total, 8 up, 8 in 2024-06-18T13:16:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-122", "mode": "writeback"}]: dispatch 2024-06-18T13:16:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-122", "mode": "writeback"}]: dispatch 2024-06-18T13:16:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-mon[37701]: pgmap v952: 236 pgs: 236 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:16:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:16:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:16:25.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-122"}]': finished 2024-06-18T13:16:25.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:25 smithi179 ceph-mon[33312]: osdmap e621: 8 total, 8 up, 8 in 2024-06-18T13:16:25.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-122", "mode": "writeback"}]: dispatch 2024-06-18T13:16:25.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-122", "mode": "writeback"}]: dispatch 2024-06-18T13:16:25.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:25 smithi179 ceph-mon[33312]: pgmap v952: 236 pgs: 236 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:16:25.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:16:25.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:16:26.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:26 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:26.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-122", "mode": "writeback"}]': finished 2024-06-18T13:16:26.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:26 smithi179 ceph-mon[33312]: osdmap e622: 8 total, 8 up, 8 in 2024-06-18T13:16:26.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:26.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:26 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-122", "mode": "writeback"}]': finished 2024-06-18T13:16:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:26 smithi167 ceph-mon[29496]: osdmap e622: 8 total, 8 up, 8 in 2024-06-18T13:16:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:26 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-122", "mode": "writeback"}]': finished 2024-06-18T13:16:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:26 smithi167 ceph-mon[37701]: osdmap e622: 8 total, 8 up, 8 in 2024-06-18T13:16:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:27.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:27.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:27 smithi179 ceph-mon[33312]: osdmap e623: 8 total, 8 up, 8 in 2024-06-18T13:16:27.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122"}]: dispatch 2024-06-18T13:16:27.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122"}]: dispatch 2024-06-18T13:16:27.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:27 smithi179 ceph-mon[33312]: pgmap v955: 236 pgs: 236 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:16:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:27 smithi167 ceph-mon[37701]: osdmap e623: 8 total, 8 up, 8 in 2024-06-18T13:16:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122"}]: dispatch 2024-06-18T13:16:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122"}]: dispatch 2024-06-18T13:16:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:27 smithi167 ceph-mon[37701]: pgmap v955: 236 pgs: 236 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:16:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:27 smithi167 ceph-mon[29496]: osdmap e623: 8 total, 8 up, 8 in 2024-06-18T13:16:27.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122"}]: dispatch 2024-06-18T13:16:27.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122"}]: dispatch 2024-06-18T13:16:27.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:27 smithi167 ceph-mon[29496]: pgmap v955: 236 pgs: 236 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:16:28.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:28 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:28.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122"}]': finished 2024-06-18T13:16:28.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:28 smithi179 ceph-mon[33312]: osdmap e624: 8 total, 8 up, 8 in 2024-06-18T13:16:28.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:28 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:28.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122"}]': finished 2024-06-18T13:16:28.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:28 smithi167 ceph-mon[37701]: osdmap e624: 8 total, 8 up, 8 in 2024-06-18T13:16:28.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:28 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:28.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-122"}]': finished 2024-06-18T13:16:28.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:28 smithi167 ceph-mon[29496]: osdmap e624: 8 total, 8 up, 8 in 2024-06-18T13:16:29.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:29 smithi179 ceph-mon[33312]: osdmap e625: 8 total, 8 up, 8 in 2024-06-18T13:16:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:29 smithi179 ceph-mon[33312]: pgmap v958: 204 pgs: 204 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:29.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:16:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:16:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:29 smithi167 ceph-mon[29496]: osdmap e625: 8 total, 8 up, 8 in 2024-06-18T13:16:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:29 smithi167 ceph-mon[29496]: pgmap v958: 204 pgs: 204 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:29 smithi167 ceph-mon[37701]: osdmap e625: 8 total, 8 up, 8 in 2024-06-18T13:16:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:29 smithi167 ceph-mon[37701]: pgmap v958: 204 pgs: 204 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:30.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:30.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:30 smithi179 ceph-mon[33312]: osdmap e626: 8 total, 8 up, 8 in 2024-06-18T13:16:30.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:30.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:30 smithi167 ceph-mon[29496]: osdmap e626: 8 total, 8 up, 8 in 2024-06-18T13:16:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:30 smithi167 ceph-mon[37701]: osdmap e626: 8 total, 8 up, 8 in 2024-06-18T13:16:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:31.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:31.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:31 smithi179 ceph-mon[33312]: osdmap e627: 8 total, 8 up, 8 in 2024-06-18T13:16:31.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:31 smithi179 ceph-mon[33312]: pgmap v961: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:16:31.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:31 smithi167 ceph-mon[29496]: osdmap e627: 8 total, 8 up, 8 in 2024-06-18T13:16:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:31 smithi167 ceph-mon[29496]: pgmap v961: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:16:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:31 smithi167 ceph-mon[37701]: osdmap e627: 8 total, 8 up, 8 in 2024-06-18T13:16:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:31 smithi167 ceph-mon[37701]: pgmap v961: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:16:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:32 smithi179 ceph-mon[33312]: osdmap e628: 8 total, 8 up, 8 in 2024-06-18T13:16:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:32.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:32 smithi167 ceph-mon[29496]: osdmap e628: 8 total, 8 up, 8 in 2024-06-18T13:16:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:32 smithi167 ceph-mon[37701]: osdmap e628: 8 total, 8 up, 8 in 2024-06-18T13:16:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:33.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:33 smithi179 ceph-mon[33312]: pgmap v963: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 253 B/s wr, 1 op/s 2024-06-18T13:16:33.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:33 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:33.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:33 smithi179 ceph-mon[33312]: osdmap e629: 8 total, 8 up, 8 in 2024-06-18T13:16:33.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-124"}]: dispatch 2024-06-18T13:16:33.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:33 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-124"}]: dispatch 2024-06-18T13:16:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:33 smithi167 ceph-mon[29496]: pgmap v963: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 253 B/s wr, 1 op/s 2024-06-18T13:16:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:33 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:33 smithi167 ceph-mon[29496]: osdmap e629: 8 total, 8 up, 8 in 2024-06-18T13:16:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-124"}]: dispatch 2024-06-18T13:16:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:33 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-124"}]: dispatch 2024-06-18T13:16:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:33 smithi167 ceph-mon[37701]: pgmap v963: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 253 B/s wr, 1 op/s 2024-06-18T13:16:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:33 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:33 smithi167 ceph-mon[37701]: osdmap e629: 8 total, 8 up, 8 in 2024-06-18T13:16:33.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-124"}]: dispatch 2024-06-18T13:16:33.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:33 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-124"}]: dispatch 2024-06-18T13:16:34.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-124"}]': finished 2024-06-18T13:16:34.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:34 smithi179 ceph-mon[33312]: osdmap e630: 8 total, 8 up, 8 in 2024-06-18T13:16:34.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-124", "mode": "writeback"}]: dispatch 2024-06-18T13:16:34.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-124", "mode": "writeback"}]: dispatch 2024-06-18T13:16:34.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:34 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:34.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-124", "mode": "writeback"}]': finished 2024-06-18T13:16:34.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:34 smithi179 ceph-mon[33312]: osdmap e631: 8 total, 8 up, 8 in 2024-06-18T13:16:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-124"}]': finished 2024-06-18T13:16:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:34 smithi167 ceph-mon[29496]: osdmap e630: 8 total, 8 up, 8 in 2024-06-18T13:16:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-124", "mode": "writeback"}]: dispatch 2024-06-18T13:16:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-124", "mode": "writeback"}]: dispatch 2024-06-18T13:16:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:34 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-124", "mode": "writeback"}]': finished 2024-06-18T13:16:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:34 smithi167 ceph-mon[29496]: osdmap e631: 8 total, 8 up, 8 in 2024-06-18T13:16:34.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-124"}]': finished 2024-06-18T13:16:34.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:34 smithi167 ceph-mon[37701]: osdmap e630: 8 total, 8 up, 8 in 2024-06-18T13:16:34.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-124", "mode": "writeback"}]: dispatch 2024-06-18T13:16:34.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-124", "mode": "writeback"}]: dispatch 2024-06-18T13:16:34.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:34 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:34.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-124", "mode": "writeback"}]': finished 2024-06-18T13:16:34.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:34 smithi167 ceph-mon[37701]: osdmap e631: 8 total, 8 up, 8 in 2024-06-18T13:16:35.348 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:16:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:16:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:16:35.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:35 smithi179 ceph-mon[33312]: pgmap v966: 236 pgs: 236 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-18T13:16:35.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:35 smithi167 ceph-mon[37701]: pgmap v966: 236 pgs: 236 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-18T13:16:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:35 smithi167 ceph-mon[29496]: pgmap v966: 236 pgs: 236 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-18T13:16:36.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:36 smithi179 ceph-mon[33312]: osdmap e632: 8 total, 8 up, 8 in 2024-06-18T13:16:36.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:36.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:36 smithi167 ceph-mon[37701]: osdmap e632: 8 total, 8 up, 8 in 2024-06-18T13:16:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:36.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:36 smithi167 ceph-mon[29496]: osdmap e632: 8 total, 8 up, 8 in 2024-06-18T13:16:36.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:36.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:37 smithi179 ceph-mon[33312]: pgmap v969: 236 pgs: 236 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-18T13:16:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:37 smithi179 ceph-mon[33312]: osdmap e633: 8 total, 8 up, 8 in 2024-06-18T13:16:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124"}]: dispatch 2024-06-18T13:16:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124"}]: dispatch 2024-06-18T13:16:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:37 smithi167 ceph-mon[37701]: pgmap v969: 236 pgs: 236 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-18T13:16:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:37 smithi167 ceph-mon[37701]: osdmap e633: 8 total, 8 up, 8 in 2024-06-18T13:16:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124"}]: dispatch 2024-06-18T13:16:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124"}]: dispatch 2024-06-18T13:16:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:37 smithi167 ceph-mon[29496]: pgmap v969: 236 pgs: 236 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-18T13:16:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:37 smithi167 ceph-mon[29496]: osdmap e633: 8 total, 8 up, 8 in 2024-06-18T13:16:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124"}]: dispatch 2024-06-18T13:16:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124"}]: dispatch 2024-06-18T13:16:38.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:38 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:38.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124"}]': finished 2024-06-18T13:16:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:38 smithi179 ceph-mon[33312]: osdmap e634: 8 total, 8 up, 8 in 2024-06-18T13:16:38.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:38 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124"}]': finished 2024-06-18T13:16:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:38 smithi167 ceph-mon[37701]: osdmap e634: 8 total, 8 up, 8 in 2024-06-18T13:16:38.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:38 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:38.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-124"}]': finished 2024-06-18T13:16:38.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:38 smithi167 ceph-mon[29496]: osdmap e634: 8 total, 8 up, 8 in 2024-06-18T13:16:39.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:39 smithi179 ceph-mon[33312]: pgmap v972: 236 pgs: 236 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:39 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:39 smithi179 ceph-mon[33312]: osdmap e635: 8 total, 8 up, 8 in 2024-06-18T13:16:39.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:16:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:16:39.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:39 smithi167 ceph-mon[37701]: pgmap v972: 236 pgs: 236 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:39 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:39 smithi167 ceph-mon[37701]: osdmap e635: 8 total, 8 up, 8 in 2024-06-18T13:16:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:39 smithi167 ceph-mon[29496]: pgmap v972: 236 pgs: 236 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:39 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:39 smithi167 ceph-mon[29496]: osdmap e635: 8 total, 8 up, 8 in 2024-06-18T13:16:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[37701]: osdmap e636: 8 total, 8 up, 8 in 2024-06-18T13:16:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:16:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[37701]: osdmap e637: 8 total, 8 up, 8 in 2024-06-18T13:16:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:40.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[29496]: osdmap e636: 8 total, 8 up, 8 in 2024-06-18T13:16:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:16:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[29496]: osdmap e637: 8 total, 8 up, 8 in 2024-06-18T13:16:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:40 smithi179 ceph-mon[33312]: osdmap e636: 8 total, 8 up, 8 in 2024-06-18T13:16:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:16:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:40 smithi179 ceph-mon[33312]: osdmap e637: 8 total, 8 up, 8 in 2024-06-18T13:16:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:41.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:41 smithi167 ceph-mon[29496]: pgmap v975: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:41.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:41.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:41.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:41 smithi167 ceph-mon[29496]: osdmap e638: 8 total, 8 up, 8 in 2024-06-18T13:16:41.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-126"}]: dispatch 2024-06-18T13:16:41.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-126"}]: dispatch 2024-06-18T13:16:41.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:41 smithi167 ceph-mon[37701]: pgmap v975: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:41.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:41.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:41.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:41 smithi167 ceph-mon[37701]: osdmap e638: 8 total, 8 up, 8 in 2024-06-18T13:16:41.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-126"}]: dispatch 2024-06-18T13:16:41.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-126"}]: dispatch 2024-06-18T13:16:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:41 smithi179 ceph-mon[33312]: pgmap v975: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:41 smithi179 ceph-mon[33312]: osdmap e638: 8 total, 8 up, 8 in 2024-06-18T13:16:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-126"}]: dispatch 2024-06-18T13:16:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-126"}]: dispatch 2024-06-18T13:16:43.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:43 smithi179 ceph-mon[33312]: pgmap v978: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:43.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-126"}]': finished 2024-06-18T13:16:43.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:43 smithi179 ceph-mon[33312]: osdmap e639: 8 total, 8 up, 8 in 2024-06-18T13:16:43.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-126", "mode": "writeback"}]: dispatch 2024-06-18T13:16:43.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-126", "mode": "writeback"}]: dispatch 2024-06-18T13:16:43.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:43 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:43.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-126", "mode": "writeback"}]': finished 2024-06-18T13:16:43.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:43 smithi179 ceph-mon[33312]: osdmap e640: 8 total, 8 up, 8 in 2024-06-18T13:16:43.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:43.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[37701]: pgmap v978: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-126"}]': finished 2024-06-18T13:16:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[37701]: osdmap e639: 8 total, 8 up, 8 in 2024-06-18T13:16:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-126", "mode": "writeback"}]: dispatch 2024-06-18T13:16:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-126", "mode": "writeback"}]: dispatch 2024-06-18T13:16:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-126", "mode": "writeback"}]': finished 2024-06-18T13:16:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[37701]: osdmap e640: 8 total, 8 up, 8 in 2024-06-18T13:16:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:43.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:43.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[29496]: pgmap v978: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:43.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-126"}]': finished 2024-06-18T13:16:43.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[29496]: osdmap e639: 8 total, 8 up, 8 in 2024-06-18T13:16:43.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-126", "mode": "writeback"}]: dispatch 2024-06-18T13:16:43.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-126", "mode": "writeback"}]: dispatch 2024-06-18T13:16:43.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:43.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-126", "mode": "writeback"}]': finished 2024-06-18T13:16:43.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[29496]: osdmap e640: 8 total, 8 up, 8 in 2024-06-18T13:16:43.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:43.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:45.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:16:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:16:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:16:45.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:45 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:45.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:45 smithi167 ceph-mon[29496]: osdmap e641: 8 total, 8 up, 8 in 2024-06-18T13:16:45.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126"}]: dispatch 2024-06-18T13:16:45.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:45 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126"}]: dispatch 2024-06-18T13:16:45.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:45 smithi167 ceph-mon[29496]: pgmap v982: 236 pgs: 236 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 520 B/s wr, 3 op/s 2024-06-18T13:16:45.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:45 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:45.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:45 smithi167 ceph-mon[37701]: osdmap e641: 8 total, 8 up, 8 in 2024-06-18T13:16:45.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126"}]: dispatch 2024-06-18T13:16:45.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:45 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126"}]: dispatch 2024-06-18T13:16:45.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:45 smithi167 ceph-mon[37701]: pgmap v982: 236 pgs: 236 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 520 B/s wr, 3 op/s 2024-06-18T13:16:45.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:45 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:45.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:45 smithi179 ceph-mon[33312]: osdmap e641: 8 total, 8 up, 8 in 2024-06-18T13:16:45.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126"}]: dispatch 2024-06-18T13:16:45.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:45 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126"}]: dispatch 2024-06-18T13:16:45.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:45 smithi179 ceph-mon[33312]: pgmap v982: 236 pgs: 236 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 520 B/s wr, 3 op/s 2024-06-18T13:16:46.244 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:16:46.245 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9042 ms) 2024-06-18T13:16:46.245 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-06-18T13:16:46.245 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:16:46.245 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6045 ms) 2024-06-18T13:16:46.245 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-06-18T13:16:46.245 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:16:46.245 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5045 ms) 2024-06-18T13:16:46.245 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-06-18T13:16:46.245 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:16:46.245 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25621 ms) 2024-06-18T13:16:46.246 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-06-18T13:16:46.246 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:16:46.246 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14064 ms) 2024-06-18T13:16:46.246 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-06-18T13:16:46.246 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12071 ms) 2024-06-18T13:16:46.246 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-06-18T13:16:46.246 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-06-18T13:16:46.246 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: 00000010 69 73 |is| 2024-06-18T13:16:46.246 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: 00000012 2024-06-18T13:16:46.246 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12063 ms) 2024-06-18T13:16:46.246 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-06-18T13:16:46.247 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: require_osd_release = squid 2024-06-18T13:16:46.247 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6028 ms) 2024-06-18T13:16:46.247 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (559191 ms total) 2024-06-18T13:16:46.247 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: 2024-06-18T13:16:46.247 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-06-18T13:16:46.247 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-06-18T13:16:46.247 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1342 ms) 2024-06-18T13:16:46.247 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-06-18T13:16:46.247 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (10728 ms) 2024-06-18T13:16:46.247 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-06-18T13:16:46.247 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18060 ms) 2024-06-18T13:16:46.247 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-06-18T13:16:46.248 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-06-18T13:16:46.248 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30132 ms total) 2024-06-18T13:16:46.248 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: 2024-06-18T13:16:46.248 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-06-18T13:16:46.248 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-06-18T13:16:46.248 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7026 ms) 2024-06-18T13:16:46.248 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-06-18T13:16:46.248 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8066 ms) 2024-06-18T13:16:46.248 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-06-18T13:16:46.248 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: waiting for scrub... 2024-06-18T13:16:46.248 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: done waiting 2024-06-18T13:16:46.248 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24635 ms) 2024-06-18T13:16:46.248 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-06-18T13:16:46.248 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9745 ms) 2024-06-18T13:16:46.249 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-06-18T13:16:46.249 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7985 ms) 2024-06-18T13:16:46.249 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-06-18T13:16:46.249 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8049 ms) 2024-06-18T13:16:46.249 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-06-18T13:16:46.249 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10106 ms) 2024-06-18T13:16:46.249 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-06-18T13:16:46.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:46 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:46.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126"}]': finished 2024-06-18T13:16:46.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:46 smithi179 ceph-mon[33312]: osdmap e642: 8 total, 8 up, 8 in 2024-06-18T13:16:46.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:46 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126"}]': finished 2024-06-18T13:16:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:46 smithi167 ceph-mon[37701]: osdmap e642: 8 total, 8 up, 8 in 2024-06-18T13:16:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:46 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-126"}]': finished 2024-06-18T13:16:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:46 smithi167 ceph-mon[29496]: osdmap e642: 8 total, 8 up, 8 in 2024-06-18T13:16:47.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:47 smithi179 ceph-mon[33312]: osdmap e643: 8 total, 8 up, 8 in 2024-06-18T13:16:47.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:47 smithi179 ceph-mon[33312]: pgmap v985: 204 pgs: 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 260 B/s wr, 1 op/s 2024-06-18T13:16:47.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:47 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:16:47.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:47 smithi167 ceph-mon[37701]: osdmap e643: 8 total, 8 up, 8 in 2024-06-18T13:16:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:47 smithi167 ceph-mon[37701]: pgmap v985: 204 pgs: 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 260 B/s wr, 1 op/s 2024-06-18T13:16:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:47 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:16:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:47 smithi167 ceph-mon[29496]: osdmap e643: 8 total, 8 up, 8 in 2024-06-18T13:16:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:47 smithi167 ceph-mon[29496]: pgmap v985: 204 pgs: 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 260 B/s wr, 1 op/s 2024-06-18T13:16:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:47 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:16:48.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:48 smithi179 ceph-mon[33312]: osdmap e644: 8 total, 8 up, 8 in 2024-06-18T13:16:48.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:48.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:48 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:16:48.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:48.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:48 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:16:48.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:48 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:16:48.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:48 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:48 smithi167 ceph-mon[37701]: osdmap e644: 8 total, 8 up, 8 in 2024-06-18T13:16:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:48 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:16:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:48 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:16:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:48 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:16:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:48 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:48 smithi167 ceph-mon[29496]: osdmap e644: 8 total, 8 up, 8 in 2024-06-18T13:16:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:48 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:16:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:16:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:48 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:16:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:48 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:16:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:48 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:49.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:16:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:16:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:49 smithi179 ceph-mon[33312]: osdmap e645: 8 total, 8 up, 8 in 2024-06-18T13:16:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:49 smithi179 ceph-mon[33312]: pgmap v988: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:49 smithi167 ceph-mon[29496]: osdmap e645: 8 total, 8 up, 8 in 2024-06-18T13:16:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:49 smithi167 ceph-mon[29496]: pgmap v988: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:49.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:16:49.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:49 smithi167 ceph-mon[37701]: osdmap e645: 8 total, 8 up, 8 in 2024-06-18T13:16:49.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:49.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:16:49.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:49 smithi167 ceph-mon[37701]: pgmap v988: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:16:49.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:16:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-128"}]: dispatch 2024-06-18T13:16:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:50 smithi179 ceph-mon[33312]: osdmap e646: 8 total, 8 up, 8 in 2024-06-18T13:16:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-128"}]: dispatch 2024-06-18T13:16:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-128"}]': finished 2024-06-18T13:16:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:50 smithi179 ceph-mon[33312]: osdmap e647: 8 total, 8 up, 8 in 2024-06-18T13:16:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-128", "mode": "writeback"}]: dispatch 2024-06-18T13:16:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-128", "mode": "writeback"}]: dispatch 2024-06-18T13:16:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-128"}]: dispatch 2024-06-18T13:16:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[37701]: osdmap e646: 8 total, 8 up, 8 in 2024-06-18T13:16:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-128"}]: dispatch 2024-06-18T13:16:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-128"}]': finished 2024-06-18T13:16:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[37701]: osdmap e647: 8 total, 8 up, 8 in 2024-06-18T13:16:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-128", "mode": "writeback"}]: dispatch 2024-06-18T13:16:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-128", "mode": "writeback"}]: dispatch 2024-06-18T13:16:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:16:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-128"}]: dispatch 2024-06-18T13:16:50.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[29496]: osdmap e646: 8 total, 8 up, 8 in 2024-06-18T13:16:50.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-128"}]: dispatch 2024-06-18T13:16:50.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-128"}]': finished 2024-06-18T13:16:50.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[29496]: osdmap e647: 8 total, 8 up, 8 in 2024-06-18T13:16:50.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-128", "mode": "writeback"}]: dispatch 2024-06-18T13:16:50.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-128", "mode": "writeback"}]: dispatch 2024-06-18T13:16:51.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:51 smithi179 ceph-mon[33312]: pgmap v991: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:51.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:51 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:51.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:51 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-128", "mode": "writeback"}]': finished 2024-06-18T13:16:51.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:51 smithi179 ceph-mon[33312]: osdmap e648: 8 total, 8 up, 8 in 2024-06-18T13:16:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:51 smithi167 ceph-mon[37701]: pgmap v991: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:51 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:51 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-128", "mode": "writeback"}]': finished 2024-06-18T13:16:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:51 smithi167 ceph-mon[37701]: osdmap e648: 8 total, 8 up, 8 in 2024-06-18T13:16:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:51 smithi167 ceph-mon[29496]: pgmap v991: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:51 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:16:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:51 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-128", "mode": "writeback"}]': finished 2024-06-18T13:16:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:51 smithi167 ceph-mon[29496]: osdmap e648: 8 total, 8 up, 8 in 2024-06-18T13:16:53.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:53 smithi179 ceph-mon[33312]: pgmap v993: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:53.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:53 smithi167 ceph-mon[37701]: pgmap v993: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:53 smithi167 ceph-mon[29496]: pgmap v993: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:16:55.373 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:16:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:16:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:16:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:55 smithi179 ceph-mon[33312]: pgmap v994: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 2 op/s 2024-06-18T13:16:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:16:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:16:55.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:55 smithi167 ceph-mon[29496]: pgmap v994: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 2 op/s 2024-06-18T13:16:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:16:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:16:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:55 smithi167 ceph-mon[37701]: pgmap v994: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 2 op/s 2024-06-18T13:16:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:16:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:16:56.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:56 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:56.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:56 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:56.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:56 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:56.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:56 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:56 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:56 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:16:57.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:57 smithi167 ceph-mon[37701]: pgmap v995: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 871 B/s rd, 290 B/s wr, 1 op/s 2024-06-18T13:16:57.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:57.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:57 smithi167 ceph-mon[37701]: osdmap e649: 8 total, 8 up, 8 in 2024-06-18T13:16:57.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:57 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128"}]: dispatch 2024-06-18T13:16:57.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:57 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128"}]: dispatch 2024-06-18T13:16:57.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:57 smithi167 ceph-mon[29496]: pgmap v995: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 871 B/s rd, 290 B/s wr, 1 op/s 2024-06-18T13:16:57.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:57.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:57 smithi167 ceph-mon[29496]: osdmap e649: 8 total, 8 up, 8 in 2024-06-18T13:16:57.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:57 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128"}]: dispatch 2024-06-18T13:16:57.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:57 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128"}]: dispatch 2024-06-18T13:16:57.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:57 smithi179 ceph-mon[33312]: pgmap v995: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 871 B/s rd, 290 B/s wr, 1 op/s 2024-06-18T13:16:57.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:16:57.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:57 smithi179 ceph-mon[33312]: osdmap e649: 8 total, 8 up, 8 in 2024-06-18T13:16:57.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:57 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128"}]: dispatch 2024-06-18T13:16:57.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:57 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128"}]: dispatch 2024-06-18T13:16:58.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:58 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:58.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128"}]': finished 2024-06-18T13:16:58.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:58 smithi167 ceph-mon[37701]: osdmap e650: 8 total, 8 up, 8 in 2024-06-18T13:16:58.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:58 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:58.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128"}]': finished 2024-06-18T13:16:58.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:58 smithi167 ceph-mon[29496]: osdmap e650: 8 total, 8 up, 8 in 2024-06-18T13:16:58.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:58 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:16:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-128"}]': finished 2024-06-18T13:16:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:58 smithi179 ceph-mon[33312]: osdmap e650: 8 total, 8 up, 8 in 2024-06-18T13:16:59.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:16:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:16:59.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:59 smithi179 ceph-mon[33312]: pgmap v998: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 871 B/s rd, 290 B/s wr, 1 op/s 2024-06-18T13:16:59.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:59 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:59.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:16:59 smithi179 ceph-mon[33312]: osdmap e651: 8 total, 8 up, 8 in 2024-06-18T13:16:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:59 smithi167 ceph-mon[37701]: pgmap v998: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 871 B/s rd, 290 B/s wr, 1 op/s 2024-06-18T13:16:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:59 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:16:59 smithi167 ceph-mon[37701]: osdmap e651: 8 total, 8 up, 8 in 2024-06-18T13:16:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:59 smithi167 ceph-mon[29496]: pgmap v998: 236 pgs: 236 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 871 B/s rd, 290 B/s wr, 1 op/s 2024-06-18T13:16:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:59 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:16:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:16:59 smithi167 ceph-mon[29496]: osdmap e651: 8 total, 8 up, 8 in 2024-06-18T13:17:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:00 smithi167 ceph-mon[37701]: osdmap e652: 8 total, 8 up, 8 in 2024-06-18T13:17:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:00 smithi167 ceph-mon[37701]: osdmap e653: 8 total, 8 up, 8 in 2024-06-18T13:17:00.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:00.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:00 smithi167 ceph-mon[29496]: osdmap e652: 8 total, 8 up, 8 in 2024-06-18T13:17:00.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:00.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:00.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:00.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:00 smithi167 ceph-mon[29496]: osdmap e653: 8 total, 8 up, 8 in 2024-06-18T13:17:00.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:00 smithi179 ceph-mon[33312]: osdmap e652: 8 total, 8 up, 8 in 2024-06-18T13:17:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:00 smithi179 ceph-mon[33312]: osdmap e653: 8 total, 8 up, 8 in 2024-06-18T13:17:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:01 smithi167 ceph-mon[37701]: pgmap v1001: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:17:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:01 smithi167 ceph-mon[37701]: osdmap e654: 8 total, 8 up, 8 in 2024-06-18T13:17:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-130"}]: dispatch 2024-06-18T13:17:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-130"}]: dispatch 2024-06-18T13:17:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:01 smithi167 ceph-mon[29496]: pgmap v1001: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:17:01.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:01.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:01.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:01.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:01 smithi167 ceph-mon[29496]: osdmap e654: 8 total, 8 up, 8 in 2024-06-18T13:17:01.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-130"}]: dispatch 2024-06-18T13:17:01.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-130"}]: dispatch 2024-06-18T13:17:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:01 smithi179 ceph-mon[33312]: pgmap v1001: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:17:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:01 smithi179 ceph-mon[33312]: osdmap e654: 8 total, 8 up, 8 in 2024-06-18T13:17:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-130"}]: dispatch 2024-06-18T13:17:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-130"}]: dispatch 2024-06-18T13:17:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:03 smithi167 ceph-mon[37701]: pgmap v1004: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:17:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:03 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-130"}]': finished 2024-06-18T13:17:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:03 smithi167 ceph-mon[37701]: osdmap e655: 8 total, 8 up, 8 in 2024-06-18T13:17:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:03 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-130", "mode": "writeback"}]: dispatch 2024-06-18T13:17:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:03 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-130", "mode": "writeback"}]: dispatch 2024-06-18T13:17:03.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:03 smithi167 ceph-mon[29496]: pgmap v1004: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:17:03.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:03 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-130"}]': finished 2024-06-18T13:17:03.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:03 smithi167 ceph-mon[29496]: osdmap e655: 8 total, 8 up, 8 in 2024-06-18T13:17:03.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:03 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-130", "mode": "writeback"}]: dispatch 2024-06-18T13:17:03.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:03 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-130", "mode": "writeback"}]: dispatch 2024-06-18T13:17:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:03 smithi179 ceph-mon[33312]: pgmap v1004: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:17:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:03 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-130"}]': finished 2024-06-18T13:17:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:03 smithi179 ceph-mon[33312]: osdmap e655: 8 total, 8 up, 8 in 2024-06-18T13:17:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:03 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-130", "mode": "writeback"}]: dispatch 2024-06-18T13:17:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:03 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-130", "mode": "writeback"}]: dispatch 2024-06-18T13:17:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:04 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:17:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-130", "mode": "writeback"}]': finished 2024-06-18T13:17:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:04 smithi167 ceph-mon[29496]: osdmap e656: 8 total, 8 up, 8 in 2024-06-18T13:17:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:04.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:04 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:17:04.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-130", "mode": "writeback"}]': finished 2024-06-18T13:17:04.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:04 smithi167 ceph-mon[37701]: osdmap e656: 8 total, 8 up, 8 in 2024-06-18T13:17:04.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:04.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:04.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:04 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:17:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-130", "mode": "writeback"}]': finished 2024-06-18T13:17:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:04 smithi179 ceph-mon[33312]: osdmap e656: 8 total, 8 up, 8 in 2024-06-18T13:17:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:05.425 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:17:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:17:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[29496]: pgmap v1007: 236 pgs: 236 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-06-18T13:17:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[29496]: osdmap e657: 8 total, 8 up, 8 in 2024-06-18T13:17:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130"}]: dispatch 2024-06-18T13:17:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130"}]: dispatch 2024-06-18T13:17:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:17:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130"}]': finished 2024-06-18T13:17:05.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[29496]: osdmap e658: 8 total, 8 up, 8 in 2024-06-18T13:17:05.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[37701]: pgmap v1007: 236 pgs: 236 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-06-18T13:17:05.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:05.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[37701]: osdmap e657: 8 total, 8 up, 8 in 2024-06-18T13:17:05.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130"}]: dispatch 2024-06-18T13:17:05.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130"}]: dispatch 2024-06-18T13:17:05.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:17:05.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130"}]': finished 2024-06-18T13:17:05.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:05 smithi167 ceph-mon[37701]: osdmap e658: 8 total, 8 up, 8 in 2024-06-18T13:17:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:05 smithi179 ceph-mon[33312]: pgmap v1007: 236 pgs: 236 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-06-18T13:17:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:05 smithi179 ceph-mon[33312]: osdmap e657: 8 total, 8 up, 8 in 2024-06-18T13:17:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130"}]: dispatch 2024-06-18T13:17:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130"}]: dispatch 2024-06-18T13:17:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:05 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:17:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-130"}]': finished 2024-06-18T13:17:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:05 smithi179 ceph-mon[33312]: osdmap e658: 8 total, 8 up, 8 in 2024-06-18T13:17:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:06 smithi167 ceph-mon[29496]: pgmap v1010: 236 pgs: 236 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-06-18T13:17:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:06 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:06 smithi167 ceph-mon[29496]: osdmap e659: 8 total, 8 up, 8 in 2024-06-18T13:17:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:06 smithi167 ceph-mon[37701]: pgmap v1010: 236 pgs: 236 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-06-18T13:17:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:06 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:06 smithi167 ceph-mon[37701]: osdmap e659: 8 total, 8 up, 8 in 2024-06-18T13:17:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:06 smithi179 ceph-mon[33312]: pgmap v1010: 236 pgs: 236 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-06-18T13:17:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:06 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:06 smithi179 ceph-mon[33312]: osdmap e659: 8 total, 8 up, 8 in 2024-06-18T13:17:08.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:08 smithi167 ceph-mon[29496]: osdmap e660: 8 total, 8 up, 8 in 2024-06-18T13:17:08.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:08.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:08 smithi167 ceph-mon[37701]: osdmap e660: 8 total, 8 up, 8 in 2024-06-18T13:17:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:08.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:08 smithi179 ceph-mon[33312]: osdmap e660: 8 total, 8 up, 8 in 2024-06-18T13:17:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:09 smithi179 ceph-mon[33312]: pgmap v1013: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:09 smithi179 ceph-mon[33312]: osdmap e661: 8 total, 8 up, 8 in 2024-06-18T13:17:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:09.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:17:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:17:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:09 smithi167 ceph-mon[37701]: pgmap v1013: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:09 smithi167 ceph-mon[37701]: osdmap e661: 8 total, 8 up, 8 in 2024-06-18T13:17:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:09.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:09.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:09 smithi167 ceph-mon[29496]: pgmap v1013: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:09 smithi167 ceph-mon[29496]: osdmap e661: 8 total, 8 up, 8 in 2024-06-18T13:17:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:09.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:10 smithi167 ceph-mon[37701]: osdmap e662: 8 total, 8 up, 8 in 2024-06-18T13:17:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:17:10.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:17:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:10.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:10 smithi167 ceph-mon[29496]: osdmap e662: 8 total, 8 up, 8 in 2024-06-18T13:17:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:17:10.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:17:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:10 smithi179 ceph-mon[33312]: osdmap e662: 8 total, 8 up, 8 in 2024-06-18T13:17:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:17:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:17:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:11 smithi167 ceph-mon[37701]: pgmap v1016: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:17:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]': finished 2024-06-18T13:17:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:11 smithi167 ceph-mon[37701]: osdmap e663: 8 total, 8 up, 8 in 2024-06-18T13:17:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-132", "mode": "writeback"}]: dispatch 2024-06-18T13:17:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-132", "mode": "writeback"}]: dispatch 2024-06-18T13:17:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:11 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:17:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-132", "mode": "writeback"}]': finished 2024-06-18T13:17:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:11 smithi167 ceph-mon[29496]: pgmap v1016: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:17:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]': finished 2024-06-18T13:17:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:11 smithi167 ceph-mon[29496]: osdmap e663: 8 total, 8 up, 8 in 2024-06-18T13:17:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-132", "mode": "writeback"}]: dispatch 2024-06-18T13:17:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-132", "mode": "writeback"}]: dispatch 2024-06-18T13:17:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:11 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:17:11.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-132", "mode": "writeback"}]': finished 2024-06-18T13:17:11.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:11 smithi179 ceph-mon[33312]: pgmap v1016: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:17:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]': finished 2024-06-18T13:17:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:11 smithi179 ceph-mon[33312]: osdmap e663: 8 total, 8 up, 8 in 2024-06-18T13:17:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-132", "mode": "writeback"}]: dispatch 2024-06-18T13:17:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-132", "mode": "writeback"}]: dispatch 2024-06-18T13:17:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:11 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:17:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-132", "mode": "writeback"}]': finished 2024-06-18T13:17:12.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:12 smithi167 ceph-mon[37701]: osdmap e664: 8 total, 8 up, 8 in 2024-06-18T13:17:12.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:12 smithi167 ceph-mon[29496]: osdmap e664: 8 total, 8 up, 8 in 2024-06-18T13:17:12.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:12 smithi179 ceph-mon[33312]: osdmap e664: 8 total, 8 up, 8 in 2024-06-18T13:17:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:13 smithi179 ceph-mon[33312]: pgmap v1019: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:13 smithi179 ceph-mon[33312]: osdmap e665: 8 total, 8 up, 8 in 2024-06-18T13:17:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:13 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:13 smithi167 ceph-mon[37701]: pgmap v1019: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:13 smithi167 ceph-mon[37701]: osdmap e665: 8 total, 8 up, 8 in 2024-06-18T13:17:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:13 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:13 smithi167 ceph-mon[29496]: pgmap v1019: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:13 smithi167 ceph-mon[29496]: osdmap e665: 8 total, 8 up, 8 in 2024-06-18T13:17:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:13 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:14 smithi179 ceph-mon[33312]: osdmap e666: 8 total, 8 up, 8 in 2024-06-18T13:17:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:14.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:14 smithi167 ceph-mon[29496]: osdmap e666: 8 total, 8 up, 8 in 2024-06-18T13:17:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:14 smithi167 ceph-mon[37701]: osdmap e666: 8 total, 8 up, 8 in 2024-06-18T13:17:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:17:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:17:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:17:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:15 smithi179 ceph-mon[33312]: pgmap v1022: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:17:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:15 smithi179 ceph-mon[33312]: osdmap e667: 8 total, 8 up, 8 in 2024-06-18T13:17:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:15 smithi167 ceph-mon[29496]: pgmap v1022: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:17:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:15 smithi167 ceph-mon[29496]: osdmap e667: 8 total, 8 up, 8 in 2024-06-18T13:17:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:15 smithi167 ceph-mon[37701]: pgmap v1022: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:17:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:15 smithi167 ceph-mon[37701]: osdmap e667: 8 total, 8 up, 8 in 2024-06-18T13:17:15.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:15.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:16.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]': finished 2024-06-18T13:17:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:16 smithi179 ceph-mon[33312]: osdmap e668: 8 total, 8 up, 8 in 2024-06-18T13:17:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]': finished 2024-06-18T13:17:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:16 smithi167 ceph-mon[37701]: osdmap e668: 8 total, 8 up, 8 in 2024-06-18T13:17:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-132"}]': finished 2024-06-18T13:17:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:16 smithi167 ceph-mon[29496]: osdmap e668: 8 total, 8 up, 8 in 2024-06-18T13:17:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:17 smithi179 ceph-mon[33312]: pgmap v1025: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:17:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:17 smithi179 ceph-mon[33312]: osdmap e669: 8 total, 8 up, 8 in 2024-06-18T13:17:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:17.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:17 smithi167 ceph-mon[37701]: pgmap v1025: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:17:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:17 smithi167 ceph-mon[37701]: osdmap e669: 8 total, 8 up, 8 in 2024-06-18T13:17:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:17 smithi167 ceph-mon[29496]: pgmap v1025: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-06-18T13:17:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:17 smithi167 ceph-mon[29496]: osdmap e669: 8 total, 8 up, 8 in 2024-06-18T13:17:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:18.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:18 smithi179 ceph-mon[33312]: osdmap e670: 8 total, 8 up, 8 in 2024-06-18T13:17:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:18 smithi179 ceph-mon[33312]: pgmap v1028: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:18 smithi167 ceph-mon[37701]: osdmap e670: 8 total, 8 up, 8 in 2024-06-18T13:17:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:18 smithi167 ceph-mon[37701]: pgmap v1028: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:18 smithi167 ceph-mon[29496]: osdmap e670: 8 total, 8 up, 8 in 2024-06-18T13:17:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132"}]: dispatch 2024-06-18T13:17:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:18 smithi167 ceph-mon[29496]: pgmap v1028: 236 pgs: 236 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:19.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:17:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:17:19.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:19 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:17:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132"}]': finished 2024-06-18T13:17:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:19 smithi179 ceph-mon[33312]: osdmap e671: 8 total, 8 up, 8 in 2024-06-18T13:17:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:19 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:19.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:19 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:17:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132"}]': finished 2024-06-18T13:17:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:19 smithi167 ceph-mon[37701]: osdmap e671: 8 total, 8 up, 8 in 2024-06-18T13:17:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:19 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:19 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:17:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-132"}]': finished 2024-06-18T13:17:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:19 smithi167 ceph-mon[29496]: osdmap e671: 8 total, 8 up, 8 in 2024-06-18T13:17:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:19 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:20.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:20 smithi179 ceph-mon[33312]: osdmap e672: 8 total, 8 up, 8 in 2024-06-18T13:17:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:20 smithi179 ceph-mon[33312]: pgmap v1031: 204 pgs: 204 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:20 smithi167 ceph-mon[37701]: osdmap e672: 8 total, 8 up, 8 in 2024-06-18T13:17:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:20 smithi167 ceph-mon[37701]: pgmap v1031: 204 pgs: 204 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:20 smithi167 ceph-mon[29496]: osdmap e672: 8 total, 8 up, 8 in 2024-06-18T13:17:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:20 smithi167 ceph-mon[29496]: pgmap v1031: 204 pgs: 204 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:21.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:21 smithi179 ceph-mon[33312]: osdmap e673: 8 total, 8 up, 8 in 2024-06-18T13:17:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:21 smithi167 ceph-mon[37701]: osdmap e673: 8 total, 8 up, 8 in 2024-06-18T13:17:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:21 smithi167 ceph-mon[29496]: osdmap e673: 8 total, 8 up, 8 in 2024-06-18T13:17:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:22 smithi179 ceph-mon[33312]: osdmap e674: 8 total, 8 up, 8 in 2024-06-18T13:17:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:22 smithi179 ceph-mon[33312]: pgmap v1034: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:22 smithi179 ceph-mon[33312]: osdmap e675: 8 total, 8 up, 8 in 2024-06-18T13:17:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-134"}]: dispatch 2024-06-18T13:17:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-134"}]: dispatch 2024-06-18T13:17:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[37701]: osdmap e674: 8 total, 8 up, 8 in 2024-06-18T13:17:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[37701]: pgmap v1034: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[37701]: osdmap e675: 8 total, 8 up, 8 in 2024-06-18T13:17:22.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-134"}]: dispatch 2024-06-18T13:17:22.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-134"}]: dispatch 2024-06-18T13:17:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[29496]: osdmap e674: 8 total, 8 up, 8 in 2024-06-18T13:17:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[29496]: pgmap v1034: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[29496]: osdmap e675: 8 total, 8 up, 8 in 2024-06-18T13:17:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-134"}]: dispatch 2024-06-18T13:17:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-134"}]: dispatch 2024-06-18T13:17:24.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-134"}]': finished 2024-06-18T13:17:24.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:24 smithi179 ceph-mon[33312]: osdmap e676: 8 total, 8 up, 8 in 2024-06-18T13:17:24.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-134", "mode": "writeback"}]: dispatch 2024-06-18T13:17:24.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-134", "mode": "writeback"}]: dispatch 2024-06-18T13:17:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-134"}]': finished 2024-06-18T13:17:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:24 smithi167 ceph-mon[37701]: osdmap e676: 8 total, 8 up, 8 in 2024-06-18T13:17:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-134", "mode": "writeback"}]: dispatch 2024-06-18T13:17:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-134", "mode": "writeback"}]: dispatch 2024-06-18T13:17:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-134"}]': finished 2024-06-18T13:17:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:24 smithi167 ceph-mon[29496]: osdmap e676: 8 total, 8 up, 8 in 2024-06-18T13:17:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-134", "mode": "writeback"}]: dispatch 2024-06-18T13:17:24.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-134", "mode": "writeback"}]: dispatch 2024-06-18T13:17:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:17:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:17:25.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:17:25.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-134", "mode": "writeback"}]': finished 2024-06-18T13:17:25.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[37701]: osdmap e677: 8 total, 8 up, 8 in 2024-06-18T13:17:25.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:25.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[37701]: pgmap v1038: 236 pgs: 236 active+clean; 583 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-18T13:17:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:17:25.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:17:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:17:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-134", "mode": "writeback"}]': finished 2024-06-18T13:17:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[29496]: osdmap e677: 8 total, 8 up, 8 in 2024-06-18T13:17:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[29496]: pgmap v1038: 236 pgs: 236 active+clean; 583 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-18T13:17:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:17:25.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:17:25.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:25 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:17:25.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-134", "mode": "writeback"}]': finished 2024-06-18T13:17:25.538 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:25 smithi179 ceph-mon[33312]: osdmap e677: 8 total, 8 up, 8 in 2024-06-18T13:17:25.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:25.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:25.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:25 smithi179 ceph-mon[33312]: pgmap v1038: 236 pgs: 236 active+clean; 583 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-18T13:17:25.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:17:25.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:17:26.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:26.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:26 smithi179 ceph-mon[33312]: osdmap e678: 8 total, 8 up, 8 in 2024-06-18T13:17:26.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134"}]: dispatch 2024-06-18T13:17:26.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:26 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134"}]: dispatch 2024-06-18T13:17:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:26 smithi167 ceph-mon[37701]: osdmap e678: 8 total, 8 up, 8 in 2024-06-18T13:17:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134"}]: dispatch 2024-06-18T13:17:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:26 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134"}]: dispatch 2024-06-18T13:17:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:26 smithi167 ceph-mon[29496]: osdmap e678: 8 total, 8 up, 8 in 2024-06-18T13:17:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134"}]: dispatch 2024-06-18T13:17:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:26 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134"}]: dispatch 2024-06-18T13:17:27.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:27 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:17:27.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134"}]': finished 2024-06-18T13:17:27.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:27 smithi179 ceph-mon[33312]: osdmap e679: 8 total, 8 up, 8 in 2024-06-18T13:17:27.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:27 smithi179 ceph-mon[33312]: pgmap v1041: 236 pgs: 236 active+clean; 583 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-18T13:17:27.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:27 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:17:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134"}]': finished 2024-06-18T13:17:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:27 smithi167 ceph-mon[37701]: osdmap e679: 8 total, 8 up, 8 in 2024-06-18T13:17:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:27 smithi167 ceph-mon[37701]: pgmap v1041: 236 pgs: 236 active+clean; 583 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-18T13:17:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:27 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:17:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-134"}]': finished 2024-06-18T13:17:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:27 smithi167 ceph-mon[29496]: osdmap e679: 8 total, 8 up, 8 in 2024-06-18T13:17:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:27 smithi167 ceph-mon[29496]: pgmap v1041: 236 pgs: 236 active+clean; 583 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-18T13:17:28.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:28 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:28.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:28 smithi179 ceph-mon[33312]: osdmap e680: 8 total, 8 up, 8 in 2024-06-18T13:17:28.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:28 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:28.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:28 smithi167 ceph-mon[37701]: osdmap e680: 8 total, 8 up, 8 in 2024-06-18T13:17:28.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:28 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:28.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:28 smithi167 ceph-mon[29496]: osdmap e680: 8 total, 8 up, 8 in 2024-06-18T13:17:29.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:17:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:17:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:29 smithi179 ceph-mon[33312]: osdmap e681: 8 total, 8 up, 8 in 2024-06-18T13:17:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:29 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:29 smithi179 ceph-mon[33312]: pgmap v1044: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 692 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:29.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:29 smithi167 ceph-mon[37701]: osdmap e681: 8 total, 8 up, 8 in 2024-06-18T13:17:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:29 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:29 smithi167 ceph-mon[37701]: pgmap v1044: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 692 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:29 smithi167 ceph-mon[29496]: osdmap e681: 8 total, 8 up, 8 in 2024-06-18T13:17:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:29 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:29 smithi167 ceph-mon[29496]: pgmap v1044: 236 pgs: 32 unknown, 204 active+clean; 583 KiB data, 692 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:30.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:30.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:30.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:30 smithi179 ceph-mon[33312]: osdmap e682: 8 total, 8 up, 8 in 2024-06-18T13:17:30.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:30 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:30.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:30 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:30 smithi167 ceph-mon[37701]: osdmap e682: 8 total, 8 up, 8 in 2024-06-18T13:17:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:30 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:30 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:30 smithi167 ceph-mon[29496]: osdmap e682: 8 total, 8 up, 8 in 2024-06-18T13:17:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:30 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:30 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:31.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:31.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:31 smithi179 ceph-mon[33312]: osdmap e683: 8 total, 8 up, 8 in 2024-06-18T13:17:31.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:31 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-136"}]: dispatch 2024-06-18T13:17:31.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-136"}]: dispatch 2024-06-18T13:17:31.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:31 smithi179 ceph-mon[33312]: pgmap v1047: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:17:31.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:31 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-136"}]': finished 2024-06-18T13:17:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:31 smithi167 ceph-mon[29496]: osdmap e683: 8 total, 8 up, 8 in 2024-06-18T13:17:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:31 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-136"}]: dispatch 2024-06-18T13:17:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-136"}]: dispatch 2024-06-18T13:17:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:31 smithi167 ceph-mon[29496]: pgmap v1047: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:17:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:31 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-136"}]': finished 2024-06-18T13:17:31.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:31.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:31 smithi167 ceph-mon[37701]: osdmap e683: 8 total, 8 up, 8 in 2024-06-18T13:17:31.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:31 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-136"}]: dispatch 2024-06-18T13:17:31.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-136"}]: dispatch 2024-06-18T13:17:31.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:31 smithi167 ceph-mon[37701]: pgmap v1047: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:17:31.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:31 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-136"}]': finished 2024-06-18T13:17:32.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:32 smithi179 ceph-mon[33312]: osdmap e684: 8 total, 8 up, 8 in 2024-06-18T13:17:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:32 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-136", "mode": "writeback"}]: dispatch 2024-06-18T13:17:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-136", "mode": "writeback"}]: dispatch 2024-06-18T13:17:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:32 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:17:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:32 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-136", "mode": "writeback"}]': finished 2024-06-18T13:17:32.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:32 smithi179 ceph-mon[33312]: osdmap e685: 8 total, 8 up, 8 in 2024-06-18T13:17:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:32 smithi167 ceph-mon[37701]: osdmap e684: 8 total, 8 up, 8 in 2024-06-18T13:17:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:32 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-136", "mode": "writeback"}]: dispatch 2024-06-18T13:17:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-136", "mode": "writeback"}]: dispatch 2024-06-18T13:17:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:32 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:17:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:32 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-136", "mode": "writeback"}]': finished 2024-06-18T13:17:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:32 smithi167 ceph-mon[37701]: osdmap e685: 8 total, 8 up, 8 in 2024-06-18T13:17:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:32 smithi167 ceph-mon[29496]: osdmap e684: 8 total, 8 up, 8 in 2024-06-18T13:17:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:32 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-136", "mode": "writeback"}]: dispatch 2024-06-18T13:17:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-136", "mode": "writeback"}]: dispatch 2024-06-18T13:17:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:32 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:17:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:32 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-136", "mode": "writeback"}]': finished 2024-06-18T13:17:32.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:32 smithi167 ceph-mon[29496]: osdmap e685: 8 total, 8 up, 8 in 2024-06-18T13:17:33.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:33 smithi179 ceph-mon[33312]: pgmap v1050: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:17:33.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:33 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:33.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:33 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:33.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:33 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:33 smithi167 ceph-mon[37701]: pgmap v1050: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:17:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:33 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:33 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:33 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:33 smithi167 ceph-mon[29496]: pgmap v1050: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-18T13:17:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:33 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:33 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:33 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:34.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:34.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:34 smithi179 ceph-mon[33312]: osdmap e686: 8 total, 8 up, 8 in 2024-06-18T13:17:34.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:34 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136"}]: dispatch 2024-06-18T13:17:34.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:34 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136"}]: dispatch 2024-06-18T13:17:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:34 smithi167 ceph-mon[37701]: osdmap e686: 8 total, 8 up, 8 in 2024-06-18T13:17:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:34 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136"}]: dispatch 2024-06-18T13:17:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:34 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136"}]: dispatch 2024-06-18T13:17:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:17:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:34 smithi167 ceph-mon[29496]: osdmap e686: 8 total, 8 up, 8 in 2024-06-18T13:17:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:34 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136"}]: dispatch 2024-06-18T13:17:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:34 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136"}]: dispatch 2024-06-18T13:17:35.334 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:17:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:17:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:17:35.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:35 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:17:35.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136"}]': finished 2024-06-18T13:17:35.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:35 smithi179 ceph-mon[33312]: osdmap e687: 8 total, 8 up, 8 in 2024-06-18T13:17:35.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:35 smithi179 ceph-mon[33312]: pgmap v1053: 236 pgs: 236 active+clean; 4.4 MiB data, 703 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-06-18T13:17:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:35 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:17:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136"}]': finished 2024-06-18T13:17:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:35 smithi167 ceph-mon[29496]: osdmap e687: 8 total, 8 up, 8 in 2024-06-18T13:17:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:35 smithi167 ceph-mon[29496]: pgmap v1053: 236 pgs: 236 active+clean; 4.4 MiB data, 703 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-06-18T13:17:35.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:35 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:17:35.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-136"}]': finished 2024-06-18T13:17:35.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:35 smithi167 ceph-mon[37701]: osdmap e687: 8 total, 8 up, 8 in 2024-06-18T13:17:35.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:35 smithi167 ceph-mon[37701]: pgmap v1053: 236 pgs: 236 active+clean; 4.4 MiB data, 703 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-06-18T13:17:36.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:36 smithi179 ceph-mon[33312]: osdmap e688: 8 total, 8 up, 8 in 2024-06-18T13:17:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:36 smithi167 ceph-mon[37701]: osdmap e688: 8 total, 8 up, 8 in 2024-06-18T13:17:36.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:36 smithi167 ceph-mon[29496]: osdmap e688: 8 total, 8 up, 8 in 2024-06-18T13:17:37.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:37 smithi179 ceph-mon[33312]: pgmap v1055: 204 pgs: 204 active+clean; 4.4 MiB data, 703 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 970 KiB/s wr, 1 op/s 2024-06-18T13:17:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:37 smithi179 ceph-mon[33312]: osdmap e689: 8 total, 8 up, 8 in 2024-06-18T13:17:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:37.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:37 smithi167 ceph-mon[37701]: pgmap v1055: 204 pgs: 204 active+clean; 4.4 MiB data, 703 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 970 KiB/s wr, 1 op/s 2024-06-18T13:17:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:37 smithi167 ceph-mon[37701]: osdmap e689: 8 total, 8 up, 8 in 2024-06-18T13:17:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:37 smithi167 ceph-mon[29496]: pgmap v1055: 204 pgs: 204 active+clean; 4.4 MiB data, 703 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 970 KiB/s wr, 1 op/s 2024-06-18T13:17:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:37 smithi167 ceph-mon[29496]: osdmap e689: 8 total, 8 up, 8 in 2024-06-18T13:17:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:38.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:38 smithi179 ceph-mon[33312]: osdmap e690: 8 total, 8 up, 8 in 2024-06-18T13:17:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:38 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:38.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:38 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:38 smithi167 ceph-mon[37701]: osdmap e690: 8 total, 8 up, 8 in 2024-06-18T13:17:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:38 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:38 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:38.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:38.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:38 smithi167 ceph-mon[29496]: osdmap e690: 8 total, 8 up, 8 in 2024-06-18T13:17:38.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:38 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:38.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:38 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:39.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:17:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:17:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:39 smithi179 ceph-mon[33312]: pgmap v1058: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 703 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:39 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:39 smithi179 ceph-mon[33312]: osdmap e691: 8 total, 8 up, 8 in 2024-06-18T13:17:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:17:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:39 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:17:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:39 smithi167 ceph-mon[37701]: pgmap v1058: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 703 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:39 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:39 smithi167 ceph-mon[37701]: osdmap e691: 8 total, 8 up, 8 in 2024-06-18T13:17:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:17:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:39 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:17:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:39 smithi167 ceph-mon[29496]: pgmap v1058: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 703 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:39 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:39 smithi167 ceph-mon[29496]: osdmap e691: 8 total, 8 up, 8 in 2024-06-18T13:17:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:17:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:39 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:17:40.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:17:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:40 smithi179 ceph-mon[33312]: osdmap e692: 8 total, 8 up, 8 in 2024-06-18T13:17:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:17:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:17:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:17:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:17:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:17:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:40 smithi167 ceph-mon[37701]: osdmap e692: 8 total, 8 up, 8 in 2024-06-18T13:17:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:17:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:17:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:17:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:17:40.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:17:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:40 smithi167 ceph-mon[29496]: osdmap e692: 8 total, 8 up, 8 in 2024-06-18T13:17:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:17:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:17:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:17:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:17:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:41 smithi167 ceph-mon[37701]: pgmap v1061: 236 pgs: 236 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:17:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:41 smithi167 ceph-mon[37701]: osdmap e693: 8 total, 8 up, 8 in 2024-06-18T13:17:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:17:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:17:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-18T13:17:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:41 smithi167 ceph-mon[37701]: osdmap e694: 8 total, 8 up, 8 in 2024-06-18T13:17:41.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:41 smithi167 ceph-mon[29496]: pgmap v1061: 236 pgs: 236 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:41.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:17:41.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:41 smithi167 ceph-mon[29496]: osdmap e693: 8 total, 8 up, 8 in 2024-06-18T13:17:41.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:17:41.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:17:41.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-18T13:17:41.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:41 smithi167 ceph-mon[29496]: osdmap e694: 8 total, 8 up, 8 in 2024-06-18T13:17:41.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:41 smithi179 ceph-mon[33312]: pgmap v1061: 236 pgs: 236 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:17:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:41 smithi179 ceph-mon[33312]: osdmap e693: 8 total, 8 up, 8 in 2024-06-18T13:17:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:17:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-18T13:17:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-18T13:17:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:41 smithi179 ceph-mon[33312]: osdmap e694: 8 total, 8 up, 8 in 2024-06-18T13:17:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:43 smithi167 ceph-mon[37701]: pgmap v1064: 236 pgs: 236 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138"}]: dispatch 2024-06-18T13:17:43.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138"}]: dispatch 2024-06-18T13:17:43.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:43 smithi167 ceph-mon[29496]: pgmap v1064: 236 pgs: 236 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:43.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:43.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:43.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138"}]: dispatch 2024-06-18T13:17:43.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138"}]: dispatch 2024-06-18T13:17:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:43 smithi179 ceph-mon[33312]: pgmap v1064: 236 pgs: 236 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:17:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138"}]: dispatch 2024-06-18T13:17:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138"}]: dispatch 2024-06-18T13:17:44.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:44 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138"}]': finished 2024-06-18T13:17:44.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:44 smithi167 ceph-mon[37701]: osdmap e695: 8 total, 8 up, 8 in 2024-06-18T13:17:44.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:44 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138"}]': finished 2024-06-18T13:17:44.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:44 smithi167 ceph-mon[29496]: osdmap e695: 8 total, 8 up, 8 in 2024-06-18T13:17:44.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:44 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-138"}]': finished 2024-06-18T13:17:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:44 smithi179 ceph-mon[33312]: osdmap e695: 8 total, 8 up, 8 in 2024-06-18T13:17:45.402 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:17:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:17:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:17:45.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:45 smithi167 ceph-mon[37701]: pgmap v1066: 236 pgs: 236 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-18T13:17:45.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:45 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:45.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:45 smithi167 ceph-mon[37701]: osdmap e696: 8 total, 8 up, 8 in 2024-06-18T13:17:45.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:45 smithi167 ceph-mon[29496]: pgmap v1066: 236 pgs: 236 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-18T13:17:45.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:45 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:45.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:45 smithi167 ceph-mon[29496]: osdmap e696: 8 total, 8 up, 8 in 2024-06-18T13:17:45.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:45 smithi179 ceph-mon[33312]: pgmap v1066: 236 pgs: 236 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-18T13:17:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:45 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:45 smithi179 ceph-mon[33312]: osdmap e696: 8 total, 8 up, 8 in 2024-06-18T13:17:46.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:46 smithi167 ceph-mon[37701]: osdmap e697: 8 total, 8 up, 8 in 2024-06-18T13:17:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:46 smithi167 ceph-mon[29496]: osdmap e697: 8 total, 8 up, 8 in 2024-06-18T13:17:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:46.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:46 smithi179 ceph-mon[33312]: osdmap e697: 8 total, 8 up, 8 in 2024-06-18T13:17:46.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:17:47.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:47 smithi167 ceph-mon[29496]: pgmap v1069: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-18T13:17:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:47 smithi167 ceph-mon[29496]: osdmap e698: 8 total, 8 up, 8 in 2024-06-18T13:17:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:47 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:47 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:17:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:47 smithi167 ceph-mon[37701]: pgmap v1069: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-18T13:17:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:47 smithi167 ceph-mon[37701]: osdmap e698: 8 total, 8 up, 8 in 2024-06-18T13:17:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:47 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:47.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:47 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:17:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:47 smithi179 ceph-mon[33312]: pgmap v1069: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-18T13:17:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:17:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:47 smithi179 ceph-mon[33312]: osdmap e698: 8 total, 8 up, 8 in 2024-06-18T13:17:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:47 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:17:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:47 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:17:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_count","val": "3"}]: dispatch 2024-06-18T13:17:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[37701]: osdmap e699: 8 total, 8 up, 8 in 2024-06-18T13:17:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_count","val": "3"}]: dispatch 2024-06-18T13:17:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:17:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:17:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:17:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_count","val": "3"}]': finished 2024-06-18T13:17:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_period","val": "3"}]: dispatch 2024-06-18T13:17:48.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[37701]: osdmap e700: 8 total, 8 up, 8 in 2024-06-18T13:17:48.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_period","val": "3"}]: dispatch 2024-06-18T13:17:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_count","val": "3"}]: dispatch 2024-06-18T13:17:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[29496]: osdmap e699: 8 total, 8 up, 8 in 2024-06-18T13:17:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_count","val": "3"}]: dispatch 2024-06-18T13:17:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:17:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:17:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:17:48.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_count","val": "3"}]': finished 2024-06-18T13:17:48.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_period","val": "3"}]: dispatch 2024-06-18T13:17:48.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[29496]: osdmap e700: 8 total, 8 up, 8 in 2024-06-18T13:17:48.729 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:48 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_period","val": "3"}]: dispatch 2024-06-18T13:17:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:17:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_count","val": "3"}]: dispatch 2024-06-18T13:17:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:48 smithi179 ceph-mon[33312]: osdmap e699: 8 total, 8 up, 8 in 2024-06-18T13:17:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_count","val": "3"}]: dispatch 2024-06-18T13:17:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:48 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:17:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:48 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:17:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:48 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:17:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_count","val": "3"}]': finished 2024-06-18T13:17:48.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:48 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_period","val": "3"}]: dispatch 2024-06-18T13:17:48.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:48 smithi179 ceph-mon[33312]: osdmap e700: 8 total, 8 up, 8 in 2024-06-18T13:17:48.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:48 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_period","val": "3"}]: dispatch 2024-06-18T13:17:49.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:17:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:17:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:49 smithi179 ceph-mon[33312]: pgmap v1072: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_period","val": "3"}]': finished 2024-06-18T13:17:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:49 smithi179 ceph-mon[33312]: osdmap e701: 8 total, 8 up, 8 in 2024-06-18T13:17:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:49 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:17:49.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:49 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:17:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:49 smithi167 ceph-mon[37701]: pgmap v1072: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_period","val": "3"}]': finished 2024-06-18T13:17:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:49 smithi167 ceph-mon[37701]: osdmap e701: 8 total, 8 up, 8 in 2024-06-18T13:17:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:49 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:17:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:49 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:17:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:49 smithi167 ceph-mon[29496]: pgmap v1072: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 708 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:17:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_period","val": "3"}]': finished 2024-06-18T13:17:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:49 smithi167 ceph-mon[29496]: osdmap e701: 8 total, 8 up, 8 in 2024-06-18T13:17:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:49 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:17:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:49 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:17:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:17:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:50 smithi167 ceph-mon[37701]: osdmap e702: 8 total, 8 up, 8 in 2024-06-18T13:17:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:50 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-18T13:17:50.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:50 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-18T13:17:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:17:50.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:50 smithi167 ceph-mon[29496]: osdmap e702: 8 total, 8 up, 8 in 2024-06-18T13:17:50.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:50 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-18T13:17:50.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:50 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-18T13:17:50.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:17:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:17:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:50 smithi179 ceph-mon[33312]: osdmap e702: 8 total, 8 up, 8 in 2024-06-18T13:17:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:50 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-18T13:17:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:50 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-18T13:17:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:51 smithi167 ceph-mon[37701]: pgmap v1075: 236 pgs: 236 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:51 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-06-18T13:17:51.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:51 smithi167 ceph-mon[37701]: osdmap e703: 8 total, 8 up, 8 in 2024-06-18T13:17:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:51 smithi167 ceph-mon[29496]: pgmap v1075: 236 pgs: 236 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:51 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-06-18T13:17:51.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:51 smithi167 ceph-mon[29496]: osdmap e703: 8 total, 8 up, 8 in 2024-06-18T13:17:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:51 smithi179 ceph-mon[33312]: pgmap v1075: 236 pgs: 236 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:51 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-06-18T13:17:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:51 smithi179 ceph-mon[33312]: osdmap e703: 8 total, 8 up, 8 in 2024-06-18T13:17:53.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:53 smithi167 ceph-mon[37701]: pgmap v1078: 236 pgs: 236 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:53.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:53 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:53 smithi167 ceph-mon[29496]: pgmap v1078: 236 pgs: 236 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:53 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:53.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:53 smithi179 ceph-mon[33312]: pgmap v1078: 236 pgs: 236 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:17:53.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:53 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:17:55.448 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:17:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:17:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:17:55.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:55 smithi167 ceph-mon[37701]: pgmap v1079: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.1 KiB/s wr, 2 op/s 2024-06-18T13:17:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:17:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:55 smithi167 ceph-mon[29496]: pgmap v1079: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.1 KiB/s wr, 2 op/s 2024-06-18T13:17:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:17:55.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:55 smithi179 ceph-mon[33312]: pgmap v1079: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.1 KiB/s wr, 2 op/s 2024-06-18T13:17:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:17:57.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:57 smithi167 ceph-mon[37701]: pgmap v1080: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 740 B/s rd, 3.5 KiB/s wr, 1 op/s 2024-06-18T13:17:57.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:57 smithi167 ceph-mon[29496]: pgmap v1080: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 740 B/s rd, 3.5 KiB/s wr, 1 op/s 2024-06-18T13:17:57.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:57 smithi179 ceph-mon[33312]: pgmap v1080: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 740 B/s rd, 3.5 KiB/s wr, 1 op/s 2024-06-18T13:17:59.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:17:59 smithi179 ceph-mon[33312]: pgmap v1081: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 3.0 KiB/s wr, 0 op/s 2024-06-18T13:17:59.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:17:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:17:59.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:17:59 smithi167 ceph-mon[37701]: pgmap v1081: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 3.0 KiB/s wr, 0 op/s 2024-06-18T13:17:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:17:59 smithi167 ceph-mon[29496]: pgmap v1081: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 3.0 KiB/s wr, 0 op/s 2024-06-18T13:18:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:01.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:01 smithi167 ceph-mon[37701]: pgmap v1082: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 6.7 KiB/s wr, 2 op/s 2024-06-18T13:18:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:01 smithi167 ceph-mon[29496]: pgmap v1082: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 6.7 KiB/s wr, 2 op/s 2024-06-18T13:18:01.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:01 smithi179 ceph-mon[33312]: pgmap v1082: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 6.7 KiB/s wr, 2 op/s 2024-06-18T13:18:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:03 smithi167 ceph-mon[37701]: pgmap v1083: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 6.0 KiB/s wr, 1 op/s 2024-06-18T13:18:03.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:03 smithi167 ceph-mon[29496]: pgmap v1083: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 6.0 KiB/s wr, 1 op/s 2024-06-18T13:18:03.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:03 smithi179 ceph-mon[33312]: pgmap v1083: 236 pgs: 236 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 6.0 KiB/s wr, 1 op/s 2024-06-18T13:18:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140"}]: dispatch 2024-06-18T13:18:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140"}]: dispatch 2024-06-18T13:18:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:04.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140"}]: dispatch 2024-06-18T13:18:04.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140"}]: dispatch 2024-06-18T13:18:04.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140"}]: dispatch 2024-06-18T13:18:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140"}]: dispatch 2024-06-18T13:18:05.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:18:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:18:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:18:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:05 smithi167 ceph-mon[29496]: pgmap v1084: 236 pgs: 236 active+clean; 4.5 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 9.0 KiB/s wr, 2 op/s 2024-06-18T13:18:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140"}]': finished 2024-06-18T13:18:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:05 smithi167 ceph-mon[29496]: osdmap e704: 8 total, 8 up, 8 in 2024-06-18T13:18:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:05 smithi167 ceph-mon[37701]: pgmap v1084: 236 pgs: 236 active+clean; 4.5 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 9.0 KiB/s wr, 2 op/s 2024-06-18T13:18:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140"}]': finished 2024-06-18T13:18:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:05 smithi167 ceph-mon[37701]: osdmap e704: 8 total, 8 up, 8 in 2024-06-18T13:18:06.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:05 smithi179 ceph-mon[33312]: pgmap v1084: 236 pgs: 236 active+clean; 4.5 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 9.0 KiB/s wr, 2 op/s 2024-06-18T13:18:06.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-140"}]': finished 2024-06-18T13:18:06.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:05 smithi179 ceph-mon[33312]: osdmap e704: 8 total, 8 up, 8 in 2024-06-18T13:18:06.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:06 smithi167 ceph-mon[37701]: osdmap e705: 8 total, 8 up, 8 in 2024-06-18T13:18:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:06 smithi167 ceph-mon[37701]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:06 smithi167 ceph-mon[37701]: osdmap e706: 8 total, 8 up, 8 in 2024-06-18T13:18:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:06 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:06 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:06 smithi167 ceph-mon[29496]: osdmap e705: 8 total, 8 up, 8 in 2024-06-18T13:18:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:06 smithi167 ceph-mon[29496]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:06 smithi167 ceph-mon[29496]: osdmap e706: 8 total, 8 up, 8 in 2024-06-18T13:18:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:06 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:06 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:06 smithi179 ceph-mon[33312]: osdmap e705: 8 total, 8 up, 8 in 2024-06-18T13:18:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:06 smithi179 ceph-mon[33312]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:06 smithi179 ceph-mon[33312]: osdmap e706: 8 total, 8 up, 8 in 2024-06-18T13:18:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:06 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:06 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:08.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:18:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:08 smithi179 ceph-mon[33312]: osdmap e707: 8 total, 8 up, 8 in 2024-06-18T13:18:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:18:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:08 smithi167 ceph-mon[37701]: osdmap e707: 8 total, 8 up, 8 in 2024-06-18T13:18:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:18:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:08 smithi167 ceph-mon[29496]: osdmap e707: 8 total, 8 up, 8 in 2024-06-18T13:18:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:09.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:18:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:18:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:09 smithi179 ceph-mon[33312]: pgmap v1090: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:18:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:18:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:09 smithi179 ceph-mon[33312]: osdmap e708: 8 total, 8 up, 8 in 2024-06-18T13:18:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:09 smithi167 ceph-mon[29496]: pgmap v1090: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:18:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:18:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:09 smithi167 ceph-mon[29496]: osdmap e708: 8 total, 8 up, 8 in 2024-06-18T13:18:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:09 smithi167 ceph-mon[37701]: pgmap v1090: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail 2024-06-18T13:18:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:18:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:09 smithi167 ceph-mon[37701]: osdmap e708: 8 total, 8 up, 8 in 2024-06-18T13:18:09.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:09.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-142"}]': finished 2024-06-18T13:18:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:10 smithi179 ceph-mon[33312]: osdmap e709: 8 total, 8 up, 8 in 2024-06-18T13:18:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-142", "mode": "writeback"}]: dispatch 2024-06-18T13:18:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-142", "mode": "writeback"}]: dispatch 2024-06-18T13:18:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:18:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:10 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:18:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-142", "mode": "writeback"}]': finished 2024-06-18T13:18:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:10 smithi179 ceph-mon[33312]: osdmap e710: 8 total, 8 up, 8 in 2024-06-18T13:18:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:18:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-142"}]': finished 2024-06-18T13:18:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[29496]: osdmap e709: 8 total, 8 up, 8 in 2024-06-18T13:18:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-142", "mode": "writeback"}]: dispatch 2024-06-18T13:18:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-142", "mode": "writeback"}]: dispatch 2024-06-18T13:18:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:18:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:18:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-142", "mode": "writeback"}]': finished 2024-06-18T13:18:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[29496]: osdmap e710: 8 total, 8 up, 8 in 2024-06-18T13:18:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:18:10.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:10.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-142"}]': finished 2024-06-18T13:18:10.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[37701]: osdmap e709: 8 total, 8 up, 8 in 2024-06-18T13:18:10.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-142", "mode": "writeback"}]: dispatch 2024-06-18T13:18:10.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-142", "mode": "writeback"}]: dispatch 2024-06-18T13:18:10.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:18:10.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:18:10.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-142", "mode": "writeback"}]': finished 2024-06-18T13:18:10.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[37701]: osdmap e710: 8 total, 8 up, 8 in 2024-06-18T13:18:10.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:18:11.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:11 smithi179 ceph-mon[33312]: pgmap v1093: 236 pgs: 236 active+clean; 4.4 MiB data, 749 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-18T13:18:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:18:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:18:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:11 smithi179 ceph-mon[33312]: osdmap e711: 8 total, 8 up, 8 in 2024-06-18T13:18:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:18:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:18:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:11 smithi167 ceph-mon[37701]: pgmap v1093: 236 pgs: 236 active+clean; 4.4 MiB data, 749 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-18T13:18:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:18:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:18:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:11 smithi167 ceph-mon[37701]: osdmap e711: 8 total, 8 up, 8 in 2024-06-18T13:18:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:18:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:18:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:11 smithi167 ceph-mon[29496]: pgmap v1093: 236 pgs: 236 active+clean; 4.4 MiB data, 749 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-18T13:18:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:18:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:18:11.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:11 smithi167 ceph-mon[29496]: osdmap e711: 8 total, 8 up, 8 in 2024-06-18T13:18:11.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:18:11.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:18:13.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:13 smithi179 ceph-mon[33312]: pgmap v1096: 236 pgs: 236 active+clean; 4.4 MiB data, 749 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-18T13:18:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:18:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:13 smithi179 ceph-mon[33312]: osdmap e712: 8 total, 8 up, 8 in 2024-06-18T13:18:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:18:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:18:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:13 smithi167 ceph-mon[37701]: pgmap v1096: 236 pgs: 236 active+clean; 4.4 MiB data, 749 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-18T13:18:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:18:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:13 smithi167 ceph-mon[37701]: osdmap e712: 8 total, 8 up, 8 in 2024-06-18T13:18:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:18:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:18:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:13 smithi167 ceph-mon[29496]: pgmap v1096: 236 pgs: 236 active+clean; 4.4 MiB data, 749 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-18T13:18:13.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:18:13.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:13 smithi167 ceph-mon[29496]: osdmap e712: 8 total, 8 up, 8 in 2024-06-18T13:18:13.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:18:13.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:18:14.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:14 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:18:14.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:18:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:14 smithi179 ceph-mon[33312]: osdmap e713: 8 total, 8 up, 8 in 2024-06-18T13:18:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:18:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:18:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-18T13:18:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:14 smithi179 ceph-mon[33312]: osdmap e714: 8 total, 8 up, 8 in 2024-06-18T13:18:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:14 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:18:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:18:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:14 smithi167 ceph-mon[37701]: osdmap e713: 8 total, 8 up, 8 in 2024-06-18T13:18:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:18:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:18:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-18T13:18:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:14 smithi167 ceph-mon[37701]: osdmap e714: 8 total, 8 up, 8 in 2024-06-18T13:18:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:14 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:18:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:18:14.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:14 smithi167 ceph-mon[29496]: osdmap e713: 8 total, 8 up, 8 in 2024-06-18T13:18:14.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:18:14.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:18:14.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-18T13:18:14.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:14 smithi167 ceph-mon[29496]: osdmap e714: 8 total, 8 up, 8 in 2024-06-18T13:18:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:18:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:18:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:18:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:15 smithi179 ceph-mon[33312]: pgmap v1099: 236 pgs: 236 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:15 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-18T13:18:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-18T13:18:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:15 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-06-18T13:18:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:15 smithi167 ceph-mon[29496]: pgmap v1099: 236 pgs: 236 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:15 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-18T13:18:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-18T13:18:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:15 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-06-18T13:18:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:15 smithi167 ceph-mon[37701]: pgmap v1099: 236 pgs: 236 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:15 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-18T13:18:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-18T13:18:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:15 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-06-18T13:18:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:16 smithi179 ceph-mon[33312]: osdmap e715: 8 total, 8 up, 8 in 2024-06-18T13:18:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-18T13:18:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-18T13:18:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:16 smithi179 ceph-mon[33312]: pgmap v1102: 236 pgs: 236 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:16.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:16 smithi167 ceph-mon[29496]: osdmap e715: 8 total, 8 up, 8 in 2024-06-18T13:18:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-18T13:18:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-18T13:18:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:16 smithi167 ceph-mon[29496]: pgmap v1102: 236 pgs: 236 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:16 smithi167 ceph-mon[37701]: osdmap e715: 8 total, 8 up, 8 in 2024-06-18T13:18:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-18T13:18:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-18T13:18:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:16 smithi167 ceph-mon[37701]: pgmap v1102: 236 pgs: 236 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-06-18T13:18:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:17 smithi179 ceph-mon[33312]: osdmap e716: 8 total, 8 up, 8 in 2024-06-18T13:18:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-06-18T13:18:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:17 smithi167 ceph-mon[37701]: osdmap e716: 8 total, 8 up, 8 in 2024-06-18T13:18:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-06-18T13:18:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:17 smithi167 ceph-mon[29496]: osdmap e716: 8 total, 8 up, 8 in 2024-06-18T13:18:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:18.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:18:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:18 smithi179 ceph-mon[33312]: osdmap e717: 8 total, 8 up, 8 in 2024-06-18T13:18:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:18 smithi179 ceph-mon[33312]: pgmap v1105: 236 pgs: 236 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:18:18.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:18:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:18 smithi167 ceph-mon[37701]: osdmap e717: 8 total, 8 up, 8 in 2024-06-18T13:18:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:18 smithi167 ceph-mon[37701]: pgmap v1105: 236 pgs: 236 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:18:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:18:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:18 smithi167 ceph-mon[29496]: osdmap e717: 8 total, 8 up, 8 in 2024-06-18T13:18:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:18 smithi167 ceph-mon[29496]: pgmap v1105: 236 pgs: 236 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:18:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]': finished 2024-06-18T13:18:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:19 smithi179 ceph-mon[33312]: osdmap e718: 8 total, 8 up, 8 in 2024-06-18T13:18:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:19 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:19 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:19 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:19.638 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:18:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:18:19.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]': finished 2024-06-18T13:18:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:19 smithi167 ceph-mon[37701]: osdmap e718: 8 total, 8 up, 8 in 2024-06-18T13:18:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:19 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:19 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:19 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]': finished 2024-06-18T13:18:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:19 smithi167 ceph-mon[29496]: osdmap e718: 8 total, 8 up, 8 in 2024-06-18T13:18:19.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:19.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:19.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:19 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:19.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:19 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-142"}]: dispatch 2024-06-18T13:18:19.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:19 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:20.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:20 smithi179 ceph-mon[33312]: osdmap e719: 8 total, 8 up, 8 in 2024-06-18T13:18:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:20 smithi179 ceph-mon[33312]: pgmap v1108: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:18:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:20 smithi167 ceph-mon[37701]: osdmap e719: 8 total, 8 up, 8 in 2024-06-18T13:18:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:20 smithi167 ceph-mon[37701]: pgmap v1108: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:18:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:20 smithi167 ceph-mon[29496]: osdmap e719: 8 total, 8 up, 8 in 2024-06-18T13:18:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:20 smithi167 ceph-mon[29496]: pgmap v1108: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:18:21.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:21 smithi179 ceph-mon[33312]: osdmap e720: 8 total, 8 up, 8 in 2024-06-18T13:18:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:21 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:21 smithi167 ceph-mon[37701]: osdmap e720: 8 total, 8 up, 8 in 2024-06-18T13:18:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:21 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:21 smithi167 ceph-mon[29496]: osdmap e720: 8 total, 8 up, 8 in 2024-06-18T13:18:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:21 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:22.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:18:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:22 smithi179 ceph-mon[33312]: osdmap e721: 8 total, 8 up, 8 in 2024-06-18T13:18:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:22 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:22 smithi179 ceph-mon[33312]: pgmap v1111: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:18:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:18:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:22 smithi167 ceph-mon[37701]: osdmap e721: 8 total, 8 up, 8 in 2024-06-18T13:18:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:22 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:22 smithi167 ceph-mon[37701]: pgmap v1111: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:18:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:18:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:22 smithi167 ceph-mon[29496]: osdmap e721: 8 total, 8 up, 8 in 2024-06-18T13:18:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:22 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:22 smithi167 ceph-mon[29496]: pgmap v1111: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:18:23.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:18:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:23 smithi179 ceph-mon[33312]: osdmap e722: 8 total, 8 up, 8 in 2024-06-18T13:18:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:18:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:23 smithi167 ceph-mon[29496]: osdmap e722: 8 total, 8 up, 8 in 2024-06-18T13:18:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:18:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:23 smithi167 ceph-mon[37701]: osdmap e722: 8 total, 8 up, 8 in 2024-06-18T13:18:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:24.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-144"}]': finished 2024-06-18T13:18:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-144", "mode": "readproxy"}]: dispatch 2024-06-18T13:18:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:24 smithi179 ceph-mon[33312]: osdmap e723: 8 total, 8 up, 8 in 2024-06-18T13:18:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-144", "mode": "readproxy"}]: dispatch 2024-06-18T13:18:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:24 smithi179 ceph-mon[33312]: pgmap v1114: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:18:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:24 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:18:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-144", "mode": "readproxy"}]': finished 2024-06-18T13:18:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:24 smithi179 ceph-mon[33312]: osdmap e724: 8 total, 8 up, 8 in 2024-06-18T13:18:24.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-144"}]': finished 2024-06-18T13:18:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-144", "mode": "readproxy"}]: dispatch 2024-06-18T13:18:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[37701]: osdmap e723: 8 total, 8 up, 8 in 2024-06-18T13:18:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-144", "mode": "readproxy"}]: dispatch 2024-06-18T13:18:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[37701]: pgmap v1114: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:18:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:18:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-144", "mode": "readproxy"}]': finished 2024-06-18T13:18:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[37701]: osdmap e724: 8 total, 8 up, 8 in 2024-06-18T13:18:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-144"}]': finished 2024-06-18T13:18:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-144", "mode": "readproxy"}]: dispatch 2024-06-18T13:18:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[29496]: osdmap e723: 8 total, 8 up, 8 in 2024-06-18T13:18:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-144", "mode": "readproxy"}]: dispatch 2024-06-18T13:18:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[29496]: pgmap v1114: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-18T13:18:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:18:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-144", "mode": "readproxy"}]': finished 2024-06-18T13:18:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:24 smithi167 ceph-mon[29496]: osdmap e724: 8 total, 8 up, 8 in 2024-06-18T13:18:25.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:18:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:18:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:18:26.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:26 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:26.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:26 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:18:26.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:26 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:26.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:26 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:18:26.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:26 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:26.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:26 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:18:27.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:27 smithi179 ceph-mon[33312]: pgmap v1116: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 214 B/s wr, 1 op/s 2024-06-18T13:18:27.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:27 smithi167 ceph-mon[37701]: pgmap v1116: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 214 B/s wr, 1 op/s 2024-06-18T13:18:27.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:27 smithi167 ceph-mon[29496]: pgmap v1116: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 214 B/s wr, 1 op/s 2024-06-18T13:18:29.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:29 smithi179 ceph-mon[33312]: pgmap v1117: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 0 op/s 2024-06-18T13:18:29.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:18:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:18:29.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:29 smithi167 ceph-mon[29496]: pgmap v1117: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 0 op/s 2024-06-18T13:18:29.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:29 smithi167 ceph-mon[37701]: pgmap v1117: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 0 op/s 2024-06-18T13:18:30.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:30.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:31 smithi167 ceph-mon[29496]: pgmap v1118: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 131 B/s wr, 1 op/s 2024-06-18T13:18:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:31 smithi167 ceph-mon[37701]: pgmap v1118: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 131 B/s wr, 1 op/s 2024-06-18T13:18:31.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:31 smithi179 ceph-mon[33312]: pgmap v1118: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 131 B/s wr, 1 op/s 2024-06-18T13:18:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:33 smithi167 ceph-mon[37701]: pgmap v1119: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 116 B/s wr, 1 op/s 2024-06-18T13:18:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:33 smithi167 ceph-mon[29496]: pgmap v1119: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 116 B/s wr, 1 op/s 2024-06-18T13:18:33.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:33 smithi179 ceph-mon[33312]: pgmap v1119: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 116 B/s wr, 1 op/s 2024-06-18T13:18:35.420 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:18:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:18:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:18:35.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:35 smithi167 ceph-mon[37701]: pgmap v1120: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:18:35.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:35 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:35.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:35 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:35 smithi167 ceph-mon[29496]: pgmap v1120: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:18:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:35 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:35.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:35 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:35 smithi179 ceph-mon[33312]: pgmap v1120: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:18:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:35 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:35 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:18:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:36 smithi167 ceph-mon[37701]: osdmap e725: 8 total, 8 up, 8 in 2024-06-18T13:18:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:36 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:36 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:36 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:18:36.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:18:36.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:36 smithi167 ceph-mon[29496]: osdmap e725: 8 total, 8 up, 8 in 2024-06-18T13:18:36.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:36 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:36.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:36 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:36.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:36 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:18:36.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:18:36.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:36 smithi179 ceph-mon[33312]: osdmap e725: 8 total, 8 up, 8 in 2024-06-18T13:18:36.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:36 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:36.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:36 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:36.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:36 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:18:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:37 smithi167 ceph-mon[37701]: pgmap v1122: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:18:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]': finished 2024-06-18T13:18:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:37 smithi167 ceph-mon[37701]: osdmap e726: 8 total, 8 up, 8 in 2024-06-18T13:18:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:37 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:37 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:37 smithi167 ceph-mon[29496]: pgmap v1122: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:18:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]': finished 2024-06-18T13:18:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:37 smithi167 ceph-mon[29496]: osdmap e726: 8 total, 8 up, 8 in 2024-06-18T13:18:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:37 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:37.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:37 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:37.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:37 smithi179 ceph-mon[33312]: pgmap v1122: 236 pgs: 236 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:18:37.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]': finished 2024-06-18T13:18:37.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:37 smithi179 ceph-mon[33312]: osdmap e726: 8 total, 8 up, 8 in 2024-06-18T13:18:37.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:37.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:37.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:37 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:37.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:37 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-144"}]: dispatch 2024-06-18T13:18:38.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:38 smithi167 ceph-mon[37701]: osdmap e727: 8 total, 8 up, 8 in 2024-06-18T13:18:38.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:38 smithi167 ceph-mon[29496]: osdmap e727: 8 total, 8 up, 8 in 2024-06-18T13:18:38.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:38 smithi179 ceph-mon[33312]: osdmap e727: 8 total, 8 up, 8 in 2024-06-18T13:18:39.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:18:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:18:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:39 smithi179 ceph-mon[33312]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:18:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:39 smithi179 ceph-mon[33312]: osdmap e728: 8 total, 8 up, 8 in 2024-06-18T13:18:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:39 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:39.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:39 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:39.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:39 smithi167 ceph-mon[37701]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:18:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:39 smithi167 ceph-mon[37701]: osdmap e728: 8 total, 8 up, 8 in 2024-06-18T13:18:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:39 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:39 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:39 smithi167 ceph-mon[29496]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:18:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:39 smithi167 ceph-mon[29496]: osdmap e728: 8 total, 8 up, 8 in 2024-06-18T13:18:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:39 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:39 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:18:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:18:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:40 smithi167 ceph-mon[37701]: osdmap e729: 8 total, 8 up, 8 in 2024-06-18T13:18:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:40 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:40 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:18:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:18:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:40 smithi167 ceph-mon[29496]: osdmap e729: 8 total, 8 up, 8 in 2024-06-18T13:18:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:40 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:40 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:40.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:18:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:18:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:40 smithi179 ceph-mon[33312]: osdmap e729: 8 total, 8 up, 8 in 2024-06-18T13:18:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:40 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:40 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:18:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:18:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:41 smithi167 ceph-mon[37701]: pgmap v1128: 236 pgs: 32 creating+peering, 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:18:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:41 smithi167 ceph-mon[37701]: osdmap e730: 8 total, 8 up, 8 in 2024-06-18T13:18:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:41 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:18:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:41 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:18:41.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:41 smithi167 ceph-mon[29496]: pgmap v1128: 236 pgs: 32 creating+peering, 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:41.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:18:41.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:41 smithi167 ceph-mon[29496]: osdmap e730: 8 total, 8 up, 8 in 2024-06-18T13:18:41.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:41 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:18:41.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:41 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:18:41.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:41 smithi179 ceph-mon[33312]: pgmap v1128: 236 pgs: 32 creating+peering, 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:18:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:41 smithi179 ceph-mon[33312]: osdmap e730: 8 total, 8 up, 8 in 2024-06-18T13:18:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:41 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:18:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:41 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:18:42.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-146"}]': finished 2024-06-18T13:18:42.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:42 smithi167 ceph-mon[37701]: osdmap e731: 8 total, 8 up, 8 in 2024-06-18T13:18:42.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-146", "mode": "writeback"}]: dispatch 2024-06-18T13:18:42.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-146", "mode": "writeback"}]: dispatch 2024-06-18T13:18:42.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-146"}]': finished 2024-06-18T13:18:42.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:42 smithi167 ceph-mon[29496]: osdmap e731: 8 total, 8 up, 8 in 2024-06-18T13:18:42.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-146", "mode": "writeback"}]: dispatch 2024-06-18T13:18:42.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-146", "mode": "writeback"}]: dispatch 2024-06-18T13:18:42.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi167-74266-111", "overlaypool": "test-rados-api-smithi167-74266-146"}]': finished 2024-06-18T13:18:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:42 smithi179 ceph-mon[33312]: osdmap e731: 8 total, 8 up, 8 in 2024-06-18T13:18:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-146", "mode": "writeback"}]: dispatch 2024-06-18T13:18:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-146", "mode": "writeback"}]: dispatch 2024-06-18T13:18:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:43 smithi179 ceph-mon[33312]: pgmap v1131: 236 pgs: 32 creating+peering, 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:43 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:18:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-146", "mode": "writeback"}]': finished 2024-06-18T13:18:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:43 smithi179 ceph-mon[33312]: osdmap e732: 8 total, 8 up, 8 in 2024-06-18T13:18:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:18:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:18:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:43 smithi167 ceph-mon[29496]: pgmap v1131: 236 pgs: 32 creating+peering, 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:43 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:18:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-146", "mode": "writeback"}]': finished 2024-06-18T13:18:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:43 smithi167 ceph-mon[29496]: osdmap e732: 8 total, 8 up, 8 in 2024-06-18T13:18:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:18:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:18:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:43 smithi167 ceph-mon[37701]: pgmap v1131: 236 pgs: 32 creating+peering, 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:18:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:43 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-18T13:18:43.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi167-74266-146", "mode": "writeback"}]': finished 2024-06-18T13:18:43.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:43 smithi167 ceph-mon[37701]: osdmap e732: 8 total, 8 up, 8 in 2024-06-18T13:18:43.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:18:43.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_count","val": "2"}]: dispatch 2024-06-18T13:18:44.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:44 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:18:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:18:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:44 smithi179 ceph-mon[33312]: osdmap e733: 8 total, 8 up, 8 in 2024-06-18T13:18:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:44 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:18:44.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:44 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:18:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:18:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:44 smithi167 ceph-mon[37701]: osdmap e733: 8 total, 8 up, 8 in 2024-06-18T13:18:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:44 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:18:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:44 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_count","val": "2"}]': finished 2024-06-18T13:18:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:18:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:44 smithi167 ceph-mon[29496]: osdmap e733: 8 total, 8 up, 8 in 2024-06-18T13:18:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:44 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_period","val": "600"}]: dispatch 2024-06-18T13:18:45.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:18:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:18:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:18:45.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:45 smithi179 ceph-mon[33312]: pgmap v1134: 236 pgs: 236 active+clean; 4.4 MiB data, 774 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-06-18T13:18:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:45 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:18:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:45 smithi179 ceph-mon[33312]: osdmap e734: 8 total, 8 up, 8 in 2024-06-18T13:18:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:18:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:45 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:18:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:45 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:18:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:45 smithi167 ceph-mon[29496]: pgmap v1134: 236 pgs: 236 active+clean; 4.4 MiB data, 774 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-06-18T13:18:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:45 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:18:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:45 smithi167 ceph-mon[29496]: osdmap e734: 8 total, 8 up, 8 in 2024-06-18T13:18:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:18:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:45 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:18:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:45 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:18:45.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:45 smithi167 ceph-mon[37701]: pgmap v1134: 236 pgs: 236 active+clean; 4.4 MiB data, 774 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-06-18T13:18:45.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:45 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_period","val": "600"}]': finished 2024-06-18T13:18:45.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:45 smithi167 ceph-mon[37701]: osdmap e734: 8 total, 8 up, 8 in 2024-06-18T13:18:45.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:18:45.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:45 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-18T13:18:45.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:45 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-18T13:18:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:18:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:46 smithi179 ceph-mon[33312]: osdmap e735: 8 total, 8 up, 8 in 2024-06-18T13:18:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:18:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:18:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-18T13:18:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:46 smithi179 ceph-mon[33312]: osdmap e736: 8 total, 8 up, 8 in 2024-06-18T13:18:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "target_max_objects","val": "1"}]: dispatch 2024-06-18T13:18:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:46 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "target_max_objects","val": "1"}]: dispatch 2024-06-18T13:18:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:18:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[37701]: osdmap e735: 8 total, 8 up, 8 in 2024-06-18T13:18:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:18:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:18:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-18T13:18:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[37701]: osdmap e736: 8 total, 8 up, 8 in 2024-06-18T13:18:46.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "target_max_objects","val": "1"}]: dispatch 2024-06-18T13:18:46.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "target_max_objects","val": "1"}]: dispatch 2024-06-18T13:18:46.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "hit_set_type","val": "bloom"}]': finished 2024-06-18T13:18:46.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[29496]: osdmap e735: 8 total, 8 up, 8 in 2024-06-18T13:18:46.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:18:46.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-18T13:18:46.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-18T13:18:46.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[29496]: osdmap e736: 8 total, 8 up, 8 in 2024-06-18T13:18:46.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "target_max_objects","val": "1"}]: dispatch 2024-06-18T13:18:46.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:46 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "target_max_objects","val": "1"}]: dispatch 2024-06-18T13:18:47.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:47 smithi179 ceph-mon[33312]: pgmap v1137: 236 pgs: 236 active+clean; 4.4 MiB data, 774 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-06-18T13:18:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:47 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "target_max_objects","val": "1"}]': finished 2024-06-18T13:18:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:47 smithi179 ceph-mon[33312]: osdmap e737: 8 total, 8 up, 8 in 2024-06-18T13:18:47.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:47 smithi167 ceph-mon[37701]: pgmap v1137: 236 pgs: 236 active+clean; 4.4 MiB data, 774 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-06-18T13:18:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:47 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "target_max_objects","val": "1"}]': finished 2024-06-18T13:18:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:47 smithi167 ceph-mon[37701]: osdmap e737: 8 total, 8 up, 8 in 2024-06-18T13:18:47.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:47 smithi167 ceph-mon[29496]: pgmap v1137: 236 pgs: 236 active+clean; 4.4 MiB data, 774 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-06-18T13:18:47.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:47 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-146","var": "target_max_objects","val": "1"}]': finished 2024-06-18T13:18:47.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:47 smithi167 ceph-mon[29496]: osdmap e737: 8 total, 8 up, 8 in 2024-06-18T13:18:48.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:48 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:18:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:48 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:48 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:48 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:48 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:48 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:18:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:48 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:48 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:48 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:48 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:48 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:18:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:48 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:48 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:48 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:48.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:48 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:49.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:18:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:18:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:49 smithi179 ceph-mon[33312]: pgmap v1140: 236 pgs: 236 active+clean; 4.4 MiB data, 774 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:18:49.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:49 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:18:49.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:49 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:18:49.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:49 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:49.638 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:49 smithi179 ceph-mon[33312]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-06-18T13:18:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:49 smithi167 ceph-mon[37701]: pgmap v1140: 236 pgs: 236 active+clean; 4.4 MiB data, 774 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:18:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:49 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:18:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:49 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:18:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:49 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:49.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:49 smithi167 ceph-mon[37701]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-06-18T13:18:49.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:49 smithi167 ceph-mon[29496]: pgmap v1140: 236 pgs: 236 active+clean; 4.4 MiB data, 774 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:18:49.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:49 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:18:49.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:49 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:18:49.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:49 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:49.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:49 smithi167 ceph-mon[29496]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-06-18T13:18:50.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:50 smithi179 ceph-mon[33312]: pgmap v1141: 236 pgs: 236 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 873 B/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:50.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:50 smithi167 ceph-mon[37701]: pgmap v1141: 236 pgs: 236 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 873 B/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:18:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:50 smithi167 ceph-mon[29496]: pgmap v1141: 236 pgs: 236 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 873 B/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:53.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:53 smithi167 ceph-mon[37701]: pgmap v1142: 236 pgs: 236 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 746 B/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:53.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:53 smithi167 ceph-mon[29496]: pgmap v1142: 236 pgs: 236 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 746 B/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:53.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:53 smithi179 ceph-mon[33312]: pgmap v1142: 236 pgs: 236 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 746 B/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:55.412 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:18:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:18:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:18:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:55 smithi167 ceph-mon[29496]: pgmap v1143: 236 pgs: 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:18:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:55 smithi167 ceph-mon[37701]: pgmap v1143: 236 pgs: 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:18:55.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:55 smithi179 ceph-mon[33312]: pgmap v1143: 236 pgs: 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:18:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:18:57.689 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:57 smithi179 ceph-mon[33312]: pgmap v1144: 236 pgs: 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:57.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:57 smithi167 ceph-mon[37701]: pgmap v1144: 236 pgs: 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:57.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:57 smithi167 ceph-mon[29496]: pgmap v1144: 236 pgs: 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:58.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:58 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:58.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:58 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:58.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:58 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:58.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:58 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:58.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:58 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:58 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:18:59.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:18:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:18:59.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:59 smithi179 ceph-mon[33312]: pgmap v1145: 236 pgs: 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 943 B/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:59.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:18:59.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:59 smithi179 ceph-mon[33312]: osdmap e738: 8 total, 8 up, 8 in 2024-06-18T13:18:59.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:59 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:18:59.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:18:59 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:18:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:59 smithi167 ceph-mon[37701]: pgmap v1145: 236 pgs: 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 943 B/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:18:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:59 smithi167 ceph-mon[37701]: osdmap e738: 8 total, 8 up, 8 in 2024-06-18T13:18:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:59 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:18:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:18:59 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:18:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:59 smithi167 ceph-mon[29496]: pgmap v1145: 236 pgs: 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 943 B/s rd, 0 B/s wr, 1 op/s 2024-06-18T13:18:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:18:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:59 smithi167 ceph-mon[29496]: osdmap e738: 8 total, 8 up, 8 in 2024-06-18T13:18:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:59 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:18:59.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:18:59 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:19:00.387 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:18:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=sqlstore.transactions t=2024-06-18T13:18:59.965984472Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-06-18T13:19:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]': finished 2024-06-18T13:19:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:00 smithi167 ceph-mon[37701]: osdmap e739: 8 total, 8 up, 8 in 2024-06-18T13:19:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:19:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:19:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]': finished 2024-06-18T13:19:00.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:00 smithi167 ceph-mon[29496]: osdmap e739: 8 total, 8 up, 8 in 2024-06-18T13:19:00.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:00.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:00.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:19:00.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:19:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]': finished 2024-06-18T13:19:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:00 smithi179 ceph-mon[33312]: osdmap e739: 8 total, 8 up, 8 in 2024-06-18T13:19:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:19:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-146"}]: dispatch 2024-06-18T13:19:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:01 smithi167 ceph-mon[37701]: pgmap v1148: 236 pgs: 236 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-18T13:19:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:01 smithi167 ceph-mon[37701]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:19:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:01 smithi167 ceph-mon[37701]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-06-18T13:19:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:01 smithi167 ceph-mon[37701]: osdmap e740: 8 total, 8 up, 8 in 2024-06-18T13:19:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:01 smithi167 ceph-mon[29496]: pgmap v1148: 236 pgs: 236 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-18T13:19:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:01 smithi167 ceph-mon[29496]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:19:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:01 smithi167 ceph-mon[29496]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-06-18T13:19:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:01 smithi167 ceph-mon[29496]: osdmap e740: 8 total, 8 up, 8 in 2024-06-18T13:19:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:01 smithi179 ceph-mon[33312]: pgmap v1148: 236 pgs: 236 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-18T13:19:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:01 smithi179 ceph-mon[33312]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:19:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:01 smithi179 ceph-mon[33312]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-06-18T13:19:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:01 smithi179 ceph-mon[33312]: osdmap e740: 8 total, 8 up, 8 in 2024-06-18T13:19:02.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:02 smithi167 ceph-mon[37701]: osdmap e741: 8 total, 8 up, 8 in 2024-06-18T13:19:02.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:02 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:02.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:02 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:02.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:02 smithi167 ceph-mon[29496]: osdmap e741: 8 total, 8 up, 8 in 2024-06-18T13:19:02.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:02 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:02.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:02 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:02 smithi179 ceph-mon[33312]: osdmap e741: 8 total, 8 up, 8 in 2024-06-18T13:19:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:02 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:02 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:03 smithi167 ceph-mon[37701]: pgmap v1151: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-18T13:19:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:03 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:19:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:03 smithi167 ceph-mon[37701]: osdmap e742: 8 total, 8 up, 8 in 2024-06-18T13:19:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:03 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:19:03.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:03 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:19:03.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:03 smithi167 ceph-mon[29496]: pgmap v1151: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-18T13:19:03.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:03 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:19:03.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:03 smithi167 ceph-mon[29496]: osdmap e742: 8 total, 8 up, 8 in 2024-06-18T13:19:03.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:03 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:19:03.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:03 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:19:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:03 smithi179 ceph-mon[33312]: pgmap v1151: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-18T13:19:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:03 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:19:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:03 smithi179 ceph-mon[33312]: osdmap e742: 8 total, 8 up, 8 in 2024-06-18T13:19:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:03 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:19:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:03 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-18T13:19:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:19:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:04 smithi167 ceph-mon[37701]: osdmap e743: 8 total, 8 up, 8 in 2024-06-18T13:19:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:04 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]: dispatch 2024-06-18T13:19:04.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:04 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]: dispatch 2024-06-18T13:19:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:19:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:04 smithi167 ceph-mon[29496]: osdmap e743: 8 total, 8 up, 8 in 2024-06-18T13:19:04.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:04 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]: dispatch 2024-06-18T13:19:04.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:04 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]: dispatch 2024-06-18T13:19:04.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148", "force_nonempty": "--force-nonempty" }]': finished 2024-06-18T13:19:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:04 smithi179 ceph-mon[33312]: osdmap e743: 8 total, 8 up, 8 in 2024-06-18T13:19:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:04 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]: dispatch 2024-06-18T13:19:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:04 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]: dispatch 2024-06-18T13:19:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:19:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:19:05.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:05 smithi179 ceph-mon[33312]: pgmap v1154: 236 pgs: 236 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-18T13:19:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]': finished 2024-06-18T13:19:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:05 smithi179 ceph-mon[33312]: osdmap e744: 8 total, 8 up, 8 in 2024-06-18T13:19:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:05 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]: dispatch 2024-06-18T13:19:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:05 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]: dispatch 2024-06-18T13:19:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-mon[29496]: pgmap v1154: 236 pgs: 236 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-18T13:19:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]': finished 2024-06-18T13:19:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-mon[29496]: osdmap e744: 8 total, 8 up, 8 in 2024-06-18T13:19:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]: dispatch 2024-06-18T13:19:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]: dispatch 2024-06-18T13:19:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-mon[37701]: pgmap v1154: 236 pgs: 236 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-18T13:19:05.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]': finished 2024-06-18T13:19:05.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-mon[37701]: osdmap e744: 8 total, 8 up, 8 in 2024-06-18T13:19:05.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:05.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:05.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]: dispatch 2024-06-18T13:19:05.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:05 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-148"}]: dispatch 2024-06-18T13:19:06.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:06 smithi179 ceph-mon[33312]: osdmap e745: 8 total, 8 up, 8 in 2024-06-18T13:19:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:06 smithi167 ceph-mon[37701]: osdmap e745: 8 total, 8 up, 8 in 2024-06-18T13:19:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:06 smithi167 ceph-mon[29496]: osdmap e745: 8 total, 8 up, 8 in 2024-06-18T13:19:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:07 smithi179 ceph-mon[33312]: pgmap v1157: 204 pgs: 204 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:19:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:07 smithi179 ceph-mon[33312]: osdmap e746: 8 total, 8 up, 8 in 2024-06-18T13:19:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:07 smithi167 ceph-mon[37701]: pgmap v1157: 204 pgs: 204 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:19:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:07 smithi167 ceph-mon[37701]: osdmap e746: 8 total, 8 up, 8 in 2024-06-18T13:19:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:07 smithi167 ceph-mon[29496]: pgmap v1157: 204 pgs: 204 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-18T13:19:07.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:07 smithi167 ceph-mon[29496]: osdmap e746: 8 total, 8 up, 8 in 2024-06-18T13:19:07.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:07.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:19:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:08 smithi179 ceph-mon[33312]: osdmap e747: 8 total, 8 up, 8 in 2024-06-18T13:19:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-150"}]: dispatch 2024-06-18T13:19:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-150"}]: dispatch 2024-06-18T13:19:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:08 smithi179 ceph-mon[33312]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:19:08.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:19:08.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:08 smithi167 ceph-mon[37701]: osdmap e747: 8 total, 8 up, 8 in 2024-06-18T13:19:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-150"}]: dispatch 2024-06-18T13:19:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-150"}]: dispatch 2024-06-18T13:19:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:08 smithi167 ceph-mon[37701]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:19:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:19:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:08 smithi167 ceph-mon[29496]: osdmap e747: 8 total, 8 up, 8 in 2024-06-18T13:19:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-150"}]: dispatch 2024-06-18T13:19:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-150"}]: dispatch 2024-06-18T13:19:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:08 smithi167 ceph-mon[29496]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:19:09.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:09 smithi179 ceph-mon[33312]: pgmap v1160: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:19:09.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:09 smithi179 ceph-mon[33312]: osdmap e748: 8 total, 8 up, 8 in 2024-06-18T13:19:09.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:19:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:19:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:09 smithi167 ceph-mon[37701]: pgmap v1160: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:19:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:09 smithi167 ceph-mon[37701]: osdmap e748: 8 total, 8 up, 8 in 2024-06-18T13:19:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:09 smithi167 ceph-mon[29496]: pgmap v1160: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:19:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:09 smithi167 ceph-mon[29496]: osdmap e748: 8 total, 8 up, 8 in 2024-06-18T13:19:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:10 smithi179 ceph-mon[33312]: osdmap e749: 8 total, 8 up, 8 in 2024-06-18T13:19:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:19:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:19:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:10 smithi167 ceph-mon[37701]: osdmap e749: 8 total, 8 up, 8 in 2024-06-18T13:19:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:19:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:19:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:10 smithi167 ceph-mon[29496]: osdmap e749: 8 total, 8 up, 8 in 2024-06-18T13:19:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:19:10.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:19:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:11 smithi179 ceph-mon[33312]: pgmap v1163: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:19:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:19:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:11 smithi179 ceph-mon[33312]: osdmap e750: 8 total, 8 up, 8 in 2024-06-18T13:19:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-152"}]: dispatch 2024-06-18T13:19:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:11 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-152"}]: dispatch 2024-06-18T13:19:11.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:11 smithi167 ceph-mon[37701]: pgmap v1163: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:19:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:19:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:11 smithi167 ceph-mon[37701]: osdmap e750: 8 total, 8 up, 8 in 2024-06-18T13:19:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-152"}]: dispatch 2024-06-18T13:19:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:11 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-152"}]: dispatch 2024-06-18T13:19:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:11 smithi167 ceph-mon[29496]: pgmap v1163: 236 pgs: 32 unknown, 204 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:19:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:19:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:11 smithi167 ceph-mon[29496]: osdmap e750: 8 total, 8 up, 8 in 2024-06-18T13:19:11.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:11.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:11.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-152"}]: dispatch 2024-06-18T13:19:11.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:11 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-152"}]: dispatch 2024-06-18T13:19:12.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:12 smithi179 ceph-mon[33312]: osdmap e751: 8 total, 8 up, 8 in 2024-06-18T13:19:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:12 smithi179 ceph-mon[33312]: pgmap v1166: 204 pgs: 204 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:19:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:12 smithi167 ceph-mon[37701]: osdmap e751: 8 total, 8 up, 8 in 2024-06-18T13:19:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:12 smithi167 ceph-mon[37701]: pgmap v1166: 204 pgs: 204 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:19:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:12 smithi167 ceph-mon[29496]: osdmap e751: 8 total, 8 up, 8 in 2024-06-18T13:19:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:12 smithi167 ceph-mon[29496]: pgmap v1166: 204 pgs: 204 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-18T13:19:13.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:13 smithi179 ceph-mon[33312]: osdmap e752: 8 total, 8 up, 8 in 2024-06-18T13:19:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:13 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:13 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:13 smithi167 ceph-mon[29496]: osdmap e752: 8 total, 8 up, 8 in 2024-06-18T13:19:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:13 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:13 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:13.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:13 smithi167 ceph-mon[37701]: osdmap e752: 8 total, 8 up, 8 in 2024-06-18T13:19:13.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:13 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:13.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:13 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-18T13:19:14.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:19:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:14 smithi179 ceph-mon[33312]: osdmap e753: 8 total, 8 up, 8 in 2024-06-18T13:19:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-111","var": "dedup_tier","val": "test-rados-api-smithi167-74266-154"}]: dispatch 2024-06-18T13:19:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-111","var": "dedup_tier","val": "test-rados-api-smithi167-74266-154"}]: dispatch 2024-06-18T13:19:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:14 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-154"}]: dispatch 2024-06-18T13:19:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:14 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-154"}]: dispatch 2024-06-18T13:19:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:14 smithi179 ceph-mon[33312]: pgmap v1169: 236 pgs: 236 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-06-18T13:19:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:19:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[29496]: osdmap e753: 8 total, 8 up, 8 in 2024-06-18T13:19:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-111","var": "dedup_tier","val": "test-rados-api-smithi167-74266-154"}]: dispatch 2024-06-18T13:19:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-111","var": "dedup_tier","val": "test-rados-api-smithi167-74266-154"}]: dispatch 2024-06-18T13:19:14.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:14.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:14.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-154"}]: dispatch 2024-06-18T13:19:14.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-154"}]: dispatch 2024-06-18T13:19:14.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[29496]: pgmap v1169: 236 pgs: 236 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-06-18T13:19:14.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi167-74266-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-18T13:19:14.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[37701]: osdmap e753: 8 total, 8 up, 8 in 2024-06-18T13:19:14.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-111","var": "dedup_tier","val": "test-rados-api-smithi167-74266-154"}]: dispatch 2024-06-18T13:19:14.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi167-74266-111","var": "dedup_tier","val": "test-rados-api-smithi167-74266-154"}]: dispatch 2024-06-18T13:19:14.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:14.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:14.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-154"}]: dispatch 2024-06-18T13:19:14.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi167-74266-111", "tierpool": "test-rados-api-smithi167-74266-154"}]: dispatch 2024-06-18T13:19:14.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:14 smithi167 ceph-mon[37701]: pgmap v1169: 236 pgs: 236 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-06-18T13:19:15.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:19:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:19:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:19:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:15 smithi179 ceph-mon[33312]: osdmap e754: 8 total, 8 up, 8 in 2024-06-18T13:19:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:15 smithi167 ceph-mon[29496]: osdmap e754: 8 total, 8 up, 8 in 2024-06-18T13:19:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:15 smithi167 ceph-mon[37701]: osdmap e754: 8 total, 8 up, 8 in 2024-06-18T13:19:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:16 smithi179 ceph-mon[33312]: osdmap e755: 8 total, 8 up, 8 in 2024-06-18T13:19:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:16 smithi179 ceph-mon[33312]: pgmap v1172: 196 pgs: 196 active+clean; 583 KiB data, 798 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:19:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:16 smithi167 ceph-mon[29496]: osdmap e755: 8 total, 8 up, 8 in 2024-06-18T13:19:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:16 smithi167 ceph-mon[29496]: pgmap v1172: 196 pgs: 196 active+clean; 583 KiB data, 798 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:19:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:16 smithi167 ceph-mon[37701]: osdmap e755: 8 total, 8 up, 8 in 2024-06-18T13:19:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:16 smithi167 ceph-mon[37701]: pgmap v1172: 196 pgs: 196 active+clean; 583 KiB data, 798 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:19:17.594 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7858 ms) 2024-06-18T13:19:17.594 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-06-18T13:19:17.594 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12163 ms) 2024-06-18T13:19:17.594 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-06-18T13:19:17.595 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8029 ms) 2024-06-18T13:19:17.595 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-06-18T13:19:17.595 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13091 ms) 2024-06-18T13:19:17.595 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-06-18T13:19:17.595 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7755 ms) 2024-06-18T13:19:17.595 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-06-18T13:19:17.595 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8048 ms) 2024-06-18T13:19:17.595 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-06-18T13:19:17.595 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: hmm, no HitSet yet 2024-06-18T13:19:17.595 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: ok, hit_set contains 336:602f83fe:::foo:head 2024-06-18T13:19:17.595 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9078 ms) 2024-06-18T13:19:17.595 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-06-18T13:19:17.596 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716672,0 2024-06-18T13:19:17.596 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: first is 1718716672 2024-06-18T13:19:17.596 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716672,0 2024-06-18T13:19:17.596 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716672,0 2024-06-18T13:19:17.596 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716672,1718716674,1718716675,0 2024-06-18T13:19:17.596 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716672,1718716674,1718716675,0 2024-06-18T13:19:17.596 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716672,1718716674,1718716675,0 2024-06-18T13:19:17.596 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716672,1718716674,1718716675,1718716677,1718716678,0 2024-06-18T13:19:17.596 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716672,1718716674,1718716675,1718716677,1718716678,0 2024-06-18T13:19:17.596 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716672,1718716674,1718716675,1718716677,1718716678,0 2024-06-18T13:19:17.596 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716672,1718716674,1718716675,1718716677,1718716678,1718716680,1718716681,0 2024-06-18T13:19:17.596 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716672,1718716674,1718716675,1718716677,1718716678,1718716680,1718716681,0 2024-06-18T13:19:17.596 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716672,1718716674,1718716675,1718716677,1718716678,1718716680,1718716681,0 2024-06-18T13:19:17.596 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: got ls 1718716675,1718716677,1718716678,1718716680,1718716681,1718716683,1718716684,0 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: first now 1718716675, trimmed 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (21071 ms) 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: foo0 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14080 ms) 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17888 ms) 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (23005 ms) 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5024 ms) 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3033 ms) 2024-06-18T13:19:17.597 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-06-18T13:19:17.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3027 ms) 2024-06-18T13:19:17.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-06-18T13:19:17.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3028 ms) 2024-06-18T13:19:17.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (231793 ms total) 2024-06-18T13:19:17.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: 2024-06-18T13:19:17.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-06-18T13:19:17.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (863419 ms total) 2024-06-18T13:19:17.598 INFO:tasks.workunit.client.0.smithi167.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-06-18T13:19:17.599 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:19:17.599 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=73948 2024-06-18T13:19:17.599 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 73948 2024-06-18T13:19:17.599 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:19:17.599 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74224 2024-06-18T13:19:17.599 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74224 2024-06-18T13:19:17.600 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:19:17.600 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74678 2024-06-18T13:19:17.600 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74678 2024-06-18T13:19:17.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:17 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:19:17.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:19:17.890 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:17 smithi179 ceph-mon[33312]: osdmap e756: 8 total, 8 up, 8 in 2024-06-18T13:19:17.890 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:17.890 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:17 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:19:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:19:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:17 smithi167 ceph-mon[37701]: osdmap e756: 8 total, 8 up, 8 in 2024-06-18T13:19:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:17 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:19:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:19:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:17 smithi167 ceph-mon[29496]: osdmap e756: 8 total, 8 up, 8 in 2024-06-18T13:19:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/286320192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]: dispatch 2024-06-18T13:19:18.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:19:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:18 smithi179 ceph-mon[33312]: osdmap e757: 8 total, 8 up, 8 in 2024-06-18T13:19:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:18 smithi179 ceph-mon[33312]: pgmap v1175: 196 pgs: 196 active+clean; 583 KiB data, 798 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:19:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:19:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:18 smithi167 ceph-mon[29496]: osdmap e757: 8 total, 8 up, 8 in 2024-06-18T13:19:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:18 smithi167 ceph-mon[29496]: pgmap v1175: 196 pgs: 196 active+clean; 583 KiB data, 798 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:19:18.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi167-74266-111"}]': finished 2024-06-18T13:19:18.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:18 smithi167 ceph-mon[37701]: osdmap e757: 8 total, 8 up, 8 in 2024-06-18T13:19:18.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:18 smithi167 ceph-mon[37701]: pgmap v1175: 196 pgs: 196 active+clean; 583 KiB data, 798 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:19:19.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:19:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:19:20.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:20 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:20 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:20 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:21.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:21 smithi179 ceph-mon[33312]: pgmap v1176: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:19:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:21 smithi167 ceph-mon[37701]: pgmap v1176: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:19:21.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:21 smithi167 ceph-mon[29496]: pgmap v1176: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:19:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:23 smithi167 ceph-mon[37701]: pgmap v1177: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 752 B/s rd, 0 op/s 2024-06-18T13:19:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:23 smithi167 ceph-mon[29496]: pgmap v1177: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 752 B/s rd, 0 op/s 2024-06-18T13:19:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:23 smithi179 ceph-mon[33312]: pgmap v1177: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 752 B/s rd, 0 op/s 2024-06-18T13:19:25.423 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:19:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:19:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:19:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:25 smithi167 ceph-mon[37701]: pgmap v1178: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:19:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:25 smithi167 ceph-mon[29496]: pgmap v1178: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:19:25.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:25 smithi179 ceph-mon[33312]: pgmap v1178: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:19:27.689 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:27 smithi179 ceph-mon[33312]: pgmap v1179: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-18T13:19:27.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:27 smithi167 ceph-mon[37701]: pgmap v1179: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-18T13:19:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:27 smithi167 ceph-mon[29496]: pgmap v1179: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-18T13:19:29.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:29 smithi179 ceph-mon[33312]: pgmap v1180: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 949 B/s rd, 0 op/s 2024-06-18T13:19:29.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:19:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:19:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:29 smithi167 ceph-mon[37701]: pgmap v1180: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 949 B/s rd, 0 op/s 2024-06-18T13:19:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:29 smithi167 ceph-mon[29496]: pgmap v1180: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 949 B/s rd, 0 op/s 2024-06-18T13:19:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:30.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:31 smithi167 ceph-mon[37701]: pgmap v1181: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:31 smithi167 ceph-mon[29496]: pgmap v1181: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:31.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:31 smithi179 ceph-mon[33312]: pgmap v1181: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:33 smithi167 ceph-mon[37701]: pgmap v1182: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:19:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:33 smithi167 ceph-mon[29496]: pgmap v1182: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:19:33.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:33 smithi179 ceph-mon[33312]: pgmap v1182: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:19:35.444 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:19:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:19:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:19:35.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:35 smithi167 ceph-mon[37701]: pgmap v1183: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:35 smithi167 ceph-mon[29496]: pgmap v1183: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:35.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:35 smithi179 ceph-mon[33312]: pgmap v1183: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:37 smithi167 ceph-mon[37701]: pgmap v1184: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:37 smithi167 ceph-mon[29496]: pgmap v1184: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:37.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:37 smithi179 ceph-mon[33312]: pgmap v1184: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:39.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:39 smithi167 ceph-mon[29496]: pgmap v1185: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:39 smithi167 ceph-mon[37701]: pgmap v1185: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:39 smithi179 ceph-mon[33312]: pgmap v1185: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:39.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:19:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:19:40.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:40.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:19:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:19:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:19:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:41 smithi167 ceph-mon[37701]: pgmap v1186: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:41.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:41 smithi167 ceph-mon[29496]: pgmap v1186: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:41 smithi179 ceph-mon[33312]: pgmap v1186: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:43.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:43 smithi167 ceph-mon[37701]: pgmap v1187: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:19:43.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:43 smithi167 ceph-mon[29496]: pgmap v1187: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:19:43.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:43 smithi179 ceph-mon[33312]: pgmap v1187: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:19:45.465 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:19:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:19:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:19:45.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:45 smithi167 ceph-mon[37701]: pgmap v1188: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:45.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:45 smithi167 ceph-mon[29496]: pgmap v1188: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:45 smithi179 ceph-mon[33312]: pgmap v1188: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:47.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:47 smithi167 ceph-mon[37701]: pgmap v1189: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:19:47.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:47 smithi167 ceph-mon[29496]: pgmap v1189: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:19:47.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:47 smithi179 ceph-mon[33312]: pgmap v1189: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:19:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:49 smithi167 ceph-mon[37701]: pgmap v1190: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:49 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:19:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:49 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:19:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:49 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:19:49.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:49 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:19:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:49 smithi167 ceph-mon[29496]: pgmap v1190: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:49.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:49 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:19:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:49 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:19:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:49 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:19:49.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:49 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:19:49.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:19:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:19:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:49 smithi179 ceph-mon[33312]: pgmap v1190: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:49 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:19:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:49 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:19:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:49 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:19:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:49 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:19:50.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:50.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:19:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:51 smithi179 ceph-mon[33312]: pgmap v1191: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:51 smithi167 ceph-mon[37701]: pgmap v1191: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:51 smithi167 ceph-mon[29496]: pgmap v1191: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:53.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:53 smithi179 ceph-mon[33312]: pgmap v1192: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:53.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:53 smithi167 ceph-mon[37701]: pgmap v1192: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:53.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:53 smithi167 ceph-mon[29496]: pgmap v1192: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:55.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:19:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:19:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:19:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:55 smithi179 ceph-mon[33312]: pgmap v1193: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:19:55.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:55 smithi167 ceph-mon[37701]: pgmap v1193: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:55.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:19:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:55 smithi167 ceph-mon[29496]: pgmap v1193: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:19:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:19:57.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:57 smithi179 ceph-mon[33312]: pgmap v1194: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:57 smithi167 ceph-mon[37701]: pgmap v1194: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:57 smithi167 ceph-mon[29496]: pgmap v1194: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:59.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:19:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:19:59.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:19:59 smithi179 ceph-mon[33312]: pgmap v1195: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:59.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:19:59 smithi167 ceph-mon[29496]: pgmap v1195: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:19:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:19:59 smithi167 ceph-mon[37701]: pgmap v1195: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:00 smithi179 ceph-mon[33312]: overall HEALTH_WARN 2 pool(s) do not have an application enabled 2024-06-18T13:20:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:00 smithi167 ceph-mon[29496]: overall HEALTH_WARN 2 pool(s) do not have an application enabled 2024-06-18T13:20:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:00 smithi167 ceph-mon[37701]: overall HEALTH_WARN 2 pool(s) do not have an application enabled 2024-06-18T13:20:01.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:01 smithi179 ceph-mon[33312]: pgmap v1196: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:01 smithi167 ceph-mon[37701]: pgmap v1196: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:01 smithi167 ceph-mon[29496]: pgmap v1196: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:03 smithi179 ceph-mon[33312]: pgmap v1197: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:03 smithi167 ceph-mon[37701]: pgmap v1197: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:03 smithi167 ceph-mon[29496]: pgmap v1197: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:05.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:20:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:20:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:20:05.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:05 smithi179 ceph-mon[33312]: pgmap v1198: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:05 smithi167 ceph-mon[37701]: pgmap v1198: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:05 smithi167 ceph-mon[29496]: pgmap v1198: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:07.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:07 smithi179 ceph-mon[33312]: pgmap v1199: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:07 smithi167 ceph-mon[37701]: pgmap v1199: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:07 smithi167 ceph-mon[29496]: pgmap v1199: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:09 smithi179 ceph-mon[33312]: pgmap v1200: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:09.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:20:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:20:09.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:09 smithi167 ceph-mon[37701]: pgmap v1200: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:09 smithi167 ceph-mon[29496]: pgmap v1200: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:20:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:20:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:20:11.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:11 smithi179 ceph-mon[33312]: pgmap v1201: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:11.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:11 smithi167 ceph-mon[37701]: pgmap v1201: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:11 smithi167 ceph-mon[29496]: pgmap v1201: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:13.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:13 smithi179 ceph-mon[33312]: pgmap v1202: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:13.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:13 smithi167 ceph-mon[37701]: pgmap v1202: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:13 smithi167 ceph-mon[29496]: pgmap v1202: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:15.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:20:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:20:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:20:15.752 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:15 smithi167 ceph-mon[37701]: pgmap v1203: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:15.752 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:15 smithi167 ceph-mon[29496]: pgmap v1203: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:15 smithi179 ceph-mon[33312]: pgmap v1203: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:17 smithi179 ceph-mon[33312]: pgmap v1204: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:17.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:17 smithi167 ceph-mon[37701]: pgmap v1204: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:17 smithi167 ceph-mon[29496]: pgmap v1204: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:18 smithi179 ceph-mon[33312]: pgmap v1205: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:18 smithi167 ceph-mon[29496]: pgmap v1205: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:18 smithi167 ceph-mon[37701]: pgmap v1205: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:19.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:19 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:19.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:20:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:20:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:19 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:19 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:20.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:20 smithi179 ceph-mon[33312]: pgmap v1206: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:20.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:20 smithi167 ceph-mon[37701]: pgmap v1206: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:20 smithi167 ceph-mon[29496]: pgmap v1206: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:23 smithi167 ceph-mon[37701]: pgmap v1207: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:23 smithi167 ceph-mon[29496]: pgmap v1207: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:23.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:23 smithi179 ceph-mon[33312]: pgmap v1207: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:25.447 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:20:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:20:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:20:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:25 smithi167 ceph-mon[37701]: pgmap v1208: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:20:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:25 smithi167 ceph-mon[29496]: pgmap v1208: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:20:25.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:25 smithi179 ceph-mon[33312]: pgmap v1208: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:20:27.704 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:27 smithi179 ceph-mon[33312]: pgmap v1209: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:27.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:27 smithi167 ceph-mon[37701]: pgmap v1209: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:27.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:27 smithi167 ceph-mon[29496]: pgmap v1209: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:29.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:29 smithi167 ceph-mon[37701]: pgmap v1210: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:29.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:29 smithi167 ceph-mon[29496]: pgmap v1210: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:29 smithi179 ceph-mon[33312]: pgmap v1210: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:29.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:20:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:20:30.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:30.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:31 smithi167 ceph-mon[37701]: pgmap v1211: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:31 smithi167 ceph-mon[29496]: pgmap v1211: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:31.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:31 smithi179 ceph-mon[33312]: pgmap v1211: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:33.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:33 smithi167 ceph-mon[37701]: pgmap v1212: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:33.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:33 smithi167 ceph-mon[29496]: pgmap v1212: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:33.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:33 smithi179 ceph-mon[33312]: pgmap v1212: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:35.467 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:20:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:20:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:20:35.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:35 smithi167 ceph-mon[37701]: pgmap v1213: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:35.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:35 smithi167 ceph-mon[29496]: pgmap v1213: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:35.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:35 smithi179 ceph-mon[33312]: pgmap v1213: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:37.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:37 smithi167 ceph-mon[37701]: pgmap v1214: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:37.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:37 smithi167 ceph-mon[29496]: pgmap v1214: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:37.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:37 smithi179 ceph-mon[33312]: pgmap v1214: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:39.726 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:39 smithi167 ceph-mon[29496]: pgmap v1215: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:39.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:39 smithi167 ceph-mon[37701]: pgmap v1215: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:39 smithi179 ceph-mon[33312]: pgmap v1215: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:39.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:20:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:20:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:20:40.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:20:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:20:41.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:41 smithi179 ceph-mon[33312]: pgmap v1216: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:41.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:41 smithi167 ceph-mon[37701]: pgmap v1216: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:41 smithi167 ceph-mon[29496]: pgmap v1216: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:43.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:43 smithi179 ceph-mon[33312]: pgmap v1217: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:20:43.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:43 smithi167 ceph-mon[37701]: pgmap v1217: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:20:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:43 smithi167 ceph-mon[29496]: pgmap v1217: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:20:45.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:20:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:20:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:20:45.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:45 smithi179 ceph-mon[33312]: pgmap v1218: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:45.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:45 smithi167 ceph-mon[37701]: pgmap v1218: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:45 smithi167 ceph-mon[29496]: pgmap v1218: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:47 smithi179 ceph-mon[33312]: pgmap v1219: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:20:47.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:47 smithi167 ceph-mon[37701]: pgmap v1219: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:20:47.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:47 smithi167 ceph-mon[29496]: pgmap v1219: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:20:49.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:49 smithi179 ceph-mon[33312]: pgmap v1220: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:49 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:20:49.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:20:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:20:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:49 smithi167 ceph-mon[29496]: pgmap v1220: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:49 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:20:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:49 smithi167 ceph-mon[37701]: pgmap v1220: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:49 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:20:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:20:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:20:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:20:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:20:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:20:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:20:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:20:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:20:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:20:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:20:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:51 smithi179 ceph-mon[33312]: pgmap v1221: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:51 smithi167 ceph-mon[37701]: pgmap v1221: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:51 smithi167 ceph-mon[29496]: pgmap v1221: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:53.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:53 smithi179 ceph-mon[33312]: pgmap v1222: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:20:53.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:53 smithi167 ceph-mon[37701]: pgmap v1222: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:20:53.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:53 smithi167 ceph-mon[29496]: pgmap v1222: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:20:55.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:20:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:20:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:20:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:55 smithi179 ceph-mon[33312]: pgmap v1223: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:20:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:55 smithi167 ceph-mon[37701]: pgmap v1223: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:20:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:55 smithi167 ceph-mon[29496]: pgmap v1223: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:20:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:20:57.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:57 smithi179 ceph-mon[33312]: pgmap v1224: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:20:57.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:57 smithi167 ceph-mon[29496]: pgmap v1224: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:20:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:57 smithi167 ceph-mon[37701]: pgmap v1224: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:20:59.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:20:59 smithi179 ceph-mon[33312]: pgmap v1225: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:59.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:20:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:20:59.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:20:59 smithi167 ceph-mon[37701]: pgmap v1225: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:20:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:20:59 smithi167 ceph-mon[29496]: pgmap v1225: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:00.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:00.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:01.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:01 smithi179 ceph-mon[33312]: pgmap v1226: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:01 smithi167 ceph-mon[37701]: pgmap v1226: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:01 smithi167 ceph-mon[29496]: pgmap v1226: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:03 smithi179 ceph-mon[33312]: pgmap v1227: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:03.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:03 smithi167 ceph-mon[29496]: pgmap v1227: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:03 smithi167 ceph-mon[37701]: pgmap v1227: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:21:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:21:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:21:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:05 smithi179 ceph-mon[33312]: pgmap v1228: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:05 smithi167 ceph-mon[37701]: pgmap v1228: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:05 smithi167 ceph-mon[29496]: pgmap v1228: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:07.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:07 smithi179 ceph-mon[33312]: pgmap v1229: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:07.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:07 smithi167 ceph-mon[37701]: pgmap v1229: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:07 smithi167 ceph-mon[29496]: pgmap v1229: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:09 smithi179 ceph-mon[33312]: pgmap v1230: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:09.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:21:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:21:09.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:09 smithi167 ceph-mon[37701]: pgmap v1230: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:09 smithi167 ceph-mon[29496]: pgmap v1230: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:21:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:21:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:21:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:11 smithi179 ceph-mon[33312]: pgmap v1231: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:11 smithi167 ceph-mon[37701]: pgmap v1231: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:11 smithi167 ceph-mon[29496]: pgmap v1231: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:13.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:13 smithi179 ceph-mon[33312]: pgmap v1232: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:13.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:13 smithi167 ceph-mon[29496]: pgmap v1232: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:13 smithi167 ceph-mon[37701]: pgmap v1232: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:21:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:21:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:21:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:15 smithi179 ceph-mon[33312]: pgmap v1233: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:15 smithi167 ceph-mon[37701]: pgmap v1233: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:15 smithi167 ceph-mon[29496]: pgmap v1233: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:16 smithi179 ceph-mon[33312]: pgmap v1234: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:16.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:16 smithi167 ceph-mon[37701]: pgmap v1234: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:16 smithi167 ceph-mon[29496]: pgmap v1234: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:19 smithi167 ceph-mon[37701]: pgmap v1235: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:19.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:19 smithi167 ceph-mon[29496]: pgmap v1235: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:19 smithi179 ceph-mon[33312]: pgmap v1235: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:19.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:21:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:21:20.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:20 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:20 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:20.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:20 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:21.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:21 smithi167 ceph-mon[37701]: pgmap v1236: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:21.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:21 smithi167 ceph-mon[29496]: pgmap v1236: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:21.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:21 smithi179 ceph-mon[33312]: pgmap v1236: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:23.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:23 smithi167 ceph-mon[37701]: pgmap v1237: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:23.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:23 smithi167 ceph-mon[29496]: pgmap v1237: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:23 smithi179 ceph-mon[33312]: pgmap v1237: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:21:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:21:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:21:25.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:25 smithi179 ceph-mon[33312]: pgmap v1238: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:21:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:25 smithi167 ceph-mon[37701]: pgmap v1238: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:21:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:25 smithi167 ceph-mon[29496]: pgmap v1238: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:21:27.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:27 smithi179 ceph-mon[33312]: pgmap v1239: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:27.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:27 smithi167 ceph-mon[37701]: pgmap v1239: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:27 smithi167 ceph-mon[29496]: pgmap v1239: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:29 smithi179 ceph-mon[33312]: pgmap v1240: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:29.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:21:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:21:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:29 smithi167 ceph-mon[37701]: pgmap v1240: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:29 smithi167 ceph-mon[29496]: pgmap v1240: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:30.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:31.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:31 smithi179 ceph-mon[33312]: pgmap v1241: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:31 smithi167 ceph-mon[37701]: pgmap v1241: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:31 smithi167 ceph-mon[29496]: pgmap v1241: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:33 smithi179 ceph-mon[33312]: pgmap v1242: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:33.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:33 smithi167 ceph-mon[37701]: pgmap v1242: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:33.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:33 smithi167 ceph-mon[29496]: pgmap v1242: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:21:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:21:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:21:35.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:35 smithi179 ceph-mon[33312]: pgmap v1243: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:35.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:35 smithi167 ceph-mon[37701]: pgmap v1243: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:35 smithi167 ceph-mon[29496]: pgmap v1243: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:37.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:37 smithi179 ceph-mon[33312]: pgmap v1244: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:37 smithi167 ceph-mon[37701]: pgmap v1244: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:37 smithi167 ceph-mon[29496]: pgmap v1244: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:39 smithi179 ceph-mon[33312]: pgmap v1245: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:39.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:21:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:21:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:39 smithi167 ceph-mon[37701]: pgmap v1245: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:39.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:39 smithi167 ceph-mon[29496]: pgmap v1245: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:21:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:21:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:21:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:41 smithi179 ceph-mon[33312]: pgmap v1246: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:41 smithi167 ceph-mon[37701]: pgmap v1246: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:41 smithi167 ceph-mon[29496]: pgmap v1246: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:43 smithi179 ceph-mon[33312]: pgmap v1247: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:43.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:43 smithi167 ceph-mon[37701]: pgmap v1247: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:43 smithi167 ceph-mon[29496]: pgmap v1247: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:21:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:21:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:21:45.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:45 smithi179 ceph-mon[33312]: pgmap v1248: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:45.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:45 smithi167 ceph-mon[37701]: pgmap v1248: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:45 smithi167 ceph-mon[29496]: pgmap v1248: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:47.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:47 smithi179 ceph-mon[33312]: pgmap v1249: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:47.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:47 smithi167 ceph-mon[29496]: pgmap v1249: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:47 smithi167 ceph-mon[37701]: pgmap v1249: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:49 smithi179 ceph-mon[33312]: pgmap v1250: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:49.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:21:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:21:49.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:49 smithi167 ceph-mon[37701]: pgmap v1250: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:49 smithi167 ceph-mon[29496]: pgmap v1250: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:21:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:21:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:21:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:21:50.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:21:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:21:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:21:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:21:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:21:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:21:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:21:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:21:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:21:51.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:51 smithi179 ceph-mon[33312]: pgmap v1251: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:51 smithi167 ceph-mon[37701]: pgmap v1251: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:51 smithi167 ceph-mon[29496]: pgmap v1251: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:53.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:53 smithi179 ceph-mon[33312]: pgmap v1252: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:53 smithi167 ceph-mon[37701]: pgmap v1252: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:53.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:53 smithi167 ceph-mon[29496]: pgmap v1252: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:55.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:21:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:21:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:21:55.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:55 smithi179 ceph-mon[33312]: pgmap v1253: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:21:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:55 smithi167 ceph-mon[37701]: pgmap v1253: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:21:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:55 smithi167 ceph-mon[29496]: pgmap v1253: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:21:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:21:57.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:57 smithi179 ceph-mon[33312]: pgmap v1254: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:57.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:57 smithi167 ceph-mon[37701]: pgmap v1254: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:57 smithi167 ceph-mon[29496]: pgmap v1254: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:59.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:21:59 smithi179 ceph-mon[33312]: pgmap v1255: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:59.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:21:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:21:59.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:21:59 smithi167 ceph-mon[37701]: pgmap v1255: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:21:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:21:59 smithi167 ceph-mon[29496]: pgmap v1255: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:00.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:01 smithi179 ceph-mon[33312]: pgmap v1256: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:01 smithi167 ceph-mon[37701]: pgmap v1256: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:01 smithi167 ceph-mon[29496]: pgmap v1256: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:02.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:02 smithi179 ceph-mon[33312]: pgmap v1257: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:02 smithi167 ceph-mon[37701]: pgmap v1257: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:02 smithi167 ceph-mon[29496]: pgmap v1257: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:05.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:22:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:22:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:22:05.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:05 smithi179 ceph-mon[33312]: pgmap v1258: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:05 smithi167 ceph-mon[37701]: pgmap v1258: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:05 smithi167 ceph-mon[29496]: pgmap v1258: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:07.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:07 smithi179 ceph-mon[33312]: pgmap v1259: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:07 smithi167 ceph-mon[37701]: pgmap v1259: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:07 smithi167 ceph-mon[29496]: pgmap v1259: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:09 smithi179 ceph-mon[33312]: pgmap v1260: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:09.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:22:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:22:09.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:09 smithi167 ceph-mon[29496]: pgmap v1260: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:09 smithi167 ceph-mon[37701]: pgmap v1260: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:22:10.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:22:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:22:11.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:11 smithi179 ceph-mon[33312]: pgmap v1261: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:11 smithi167 ceph-mon[29496]: pgmap v1261: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:11 smithi167 ceph-mon[37701]: pgmap v1261: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:13.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:13 smithi179 ceph-mon[33312]: pgmap v1262: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:13 smithi167 ceph-mon[29496]: pgmap v1262: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:13 smithi167 ceph-mon[37701]: pgmap v1262: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:22:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:22:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:22:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:15 smithi179 ceph-mon[33312]: pgmap v1263: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:15 smithi167 ceph-mon[37701]: pgmap v1263: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:15 smithi167 ceph-mon[29496]: pgmap v1263: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:17 smithi179 ceph-mon[33312]: pgmap v1264: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:17 smithi167 ceph-mon[37701]: pgmap v1264: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:17 smithi167 ceph-mon[29496]: pgmap v1264: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:19 smithi179 ceph-mon[33312]: pgmap v1265: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:19.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:22:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:22:19.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:19 smithi167 ceph-mon[37701]: pgmap v1265: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:19 smithi167 ceph-mon[29496]: pgmap v1265: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:21 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:21 smithi179 ceph-mon[33312]: pgmap v1266: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:21 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:21 smithi167 ceph-mon[37701]: pgmap v1266: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:21 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:21 smithi167 ceph-mon[29496]: pgmap v1266: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:23.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:23 smithi179 ceph-mon[33312]: pgmap v1267: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:22:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:23 smithi167 ceph-mon[37701]: pgmap v1267: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:22:23.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:23 smithi167 ceph-mon[29496]: pgmap v1267: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:22:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:22:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:22:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:22:25.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:25 smithi179 ceph-mon[33312]: pgmap v1268: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:22:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:25 smithi167 ceph-mon[37701]: pgmap v1268: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:22:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:25 smithi167 ceph-mon[29496]: pgmap v1268: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:22:27.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:27 smithi179 ceph-mon[33312]: pgmap v1269: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:22:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:27 smithi167 ceph-mon[37701]: pgmap v1269: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:22:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:27 smithi167 ceph-mon[29496]: pgmap v1269: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:22:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:29 smithi179 ceph-mon[33312]: pgmap v1270: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:22:29.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:22:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:22:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:29 smithi167 ceph-mon[37701]: pgmap v1270: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:22:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:29 smithi167 ceph-mon[29496]: pgmap v1270: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:22:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:31 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:31 smithi179 ceph-mon[33312]: pgmap v1271: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:31 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:31 smithi167 ceph-mon[37701]: pgmap v1271: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:31 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:31 smithi167 ceph-mon[29496]: pgmap v1271: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:33 smithi179 ceph-mon[33312]: pgmap v1272: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:22:33.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:33 smithi167 ceph-mon[37701]: pgmap v1272: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:22:33.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:33 smithi167 ceph-mon[29496]: pgmap v1272: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:22:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:22:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:22:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:22:35.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:35 smithi179 ceph-mon[33312]: pgmap v1273: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:35 smithi167 ceph-mon[29496]: pgmap v1273: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:35.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:35 smithi167 ceph-mon[37701]: pgmap v1273: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:37.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:37 smithi179 ceph-mon[33312]: pgmap v1274: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:22:37.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:37 smithi167 ceph-mon[37701]: pgmap v1274: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:22:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:37 smithi167 ceph-mon[29496]: pgmap v1274: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:22:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:39 smithi179 ceph-mon[33312]: pgmap v1275: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:39.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:22:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:22:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:39 smithi167 ceph-mon[37701]: pgmap v1275: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:39.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:39 smithi167 ceph-mon[29496]: pgmap v1275: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:22:40.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:22:40.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:22:41.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:41 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:41 smithi179 ceph-mon[33312]: pgmap v1276: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:41 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:41 smithi167 ceph-mon[37701]: pgmap v1276: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:41 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:41 smithi167 ceph-mon[29496]: pgmap v1276: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:43 smithi179 ceph-mon[33312]: pgmap v1277: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:43 smithi167 ceph-mon[29496]: pgmap v1277: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:43 smithi167 ceph-mon[37701]: pgmap v1277: 196 pgs: 196 active+clean; 583 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:22:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:22:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:22:45.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:45 smithi179 ceph-mon[33312]: pgmap v1278: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:45 smithi167 ceph-mon[29496]: pgmap v1278: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:45.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:45 smithi167 ceph-mon[37701]: pgmap v1278: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:47 smithi179 ceph-mon[33312]: pgmap v1279: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:47 smithi167 ceph-mon[37701]: pgmap v1279: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:47.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:47 smithi167 ceph-mon[29496]: pgmap v1279: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:49.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:49 smithi179 ceph-mon[33312]: pgmap v1280: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:49.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:22:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:22:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:49 smithi167 ceph-mon[37701]: pgmap v1280: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:49 smithi167 ceph-mon[29496]: pgmap v1280: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:22:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:22:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:22:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:50 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:22:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:22:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:22:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:22:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:50 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:22:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:22:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:22:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:22:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:50 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:22:51.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:51 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:51 smithi179 ceph-mon[33312]: pgmap v1281: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:51 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:51 smithi167 ceph-mon[37701]: pgmap v1281: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:51 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:22:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:51 smithi167 ceph-mon[29496]: pgmap v1281: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:22:52.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:52 smithi179 ceph-mon[33312]: pgmap v1282: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:52 smithi167 ceph-mon[37701]: pgmap v1282: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:52 smithi167 ceph-mon[29496]: pgmap v1282: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:22:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:22:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:22:55.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:55 smithi179 ceph-mon[33312]: pgmap v1283: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:22:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:55 smithi167 ceph-mon[37701]: pgmap v1283: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:22:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:55 smithi167 ceph-mon[29496]: pgmap v1283: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:22:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:22:57.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:57 smithi179 ceph-mon[33312]: pgmap v1284: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:57 smithi167 ceph-mon[37701]: pgmap v1284: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:58.000 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:57 smithi167 ceph-mon[29496]: pgmap v1284: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:59.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:22:59 smithi179 ceph-mon[33312]: pgmap v1285: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:59.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:22:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:22:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:22:59 smithi167 ceph-mon[37701]: pgmap v1285: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:22:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:22:59 smithi167 ceph-mon[29496]: pgmap v1285: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:01.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:01 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:01 smithi179 ceph-mon[33312]: pgmap v1286: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:01 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:01 smithi167 ceph-mon[37701]: pgmap v1286: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:01 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:01 smithi167 ceph-mon[29496]: pgmap v1286: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:03.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:03 smithi179 ceph-mon[33312]: pgmap v1287: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:03 smithi167 ceph-mon[37701]: pgmap v1287: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:03 smithi167 ceph-mon[29496]: pgmap v1287: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:23:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:23:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:23:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:05 smithi179 ceph-mon[33312]: pgmap v1288: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:05 smithi167 ceph-mon[37701]: pgmap v1288: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:05 smithi167 ceph-mon[29496]: pgmap v1288: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:07.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:07 smithi179 ceph-mon[33312]: pgmap v1289: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:07 smithi167 ceph-mon[37701]: pgmap v1289: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:07 smithi167 ceph-mon[29496]: pgmap v1289: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:09 smithi179 ceph-mon[33312]: pgmap v1290: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:09.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:23:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:23:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:09 smithi167 ceph-mon[37701]: pgmap v1290: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:09 smithi167 ceph-mon[29496]: pgmap v1290: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:10.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:23:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:23:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:23:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:11 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:11 smithi179 ceph-mon[33312]: pgmap v1291: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:11 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:11 smithi167 ceph-mon[37701]: pgmap v1291: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:11 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:11 smithi167 ceph-mon[29496]: pgmap v1291: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:13 smithi179 ceph-mon[33312]: pgmap v1292: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:13 smithi167 ceph-mon[37701]: pgmap v1292: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:13 smithi167 ceph-mon[29496]: pgmap v1292: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:23:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:23:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:23:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:15 smithi179 ceph-mon[33312]: pgmap v1293: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:15.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:15 smithi167 ceph-mon[37701]: pgmap v1293: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:15 smithi167 ceph-mon[29496]: pgmap v1293: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:17 smithi179 ceph-mon[33312]: pgmap v1294: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:17 smithi167 ceph-mon[37701]: pgmap v1294: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:17 smithi167 ceph-mon[29496]: pgmap v1294: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:19 smithi179 ceph-mon[33312]: pgmap v1295: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:19.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:23:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:23:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:19 smithi167 ceph-mon[37701]: pgmap v1295: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:19 smithi167 ceph-mon[29496]: pgmap v1295: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:21 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:21 smithi179 ceph-mon[33312]: pgmap v1296: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:21.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:21 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:21 smithi167 ceph-mon[37701]: pgmap v1296: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:21 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:21 smithi167 ceph-mon[29496]: pgmap v1296: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:23 smithi179 ceph-mon[33312]: pgmap v1297: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:23 smithi167 ceph-mon[29496]: pgmap v1297: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:23 smithi167 ceph-mon[37701]: pgmap v1297: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:23:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:23:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:23:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:25 smithi179 ceph-mon[33312]: pgmap v1298: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:23:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:25 smithi167 ceph-mon[37701]: pgmap v1298: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:23:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:25 smithi167 ceph-mon[29496]: pgmap v1298: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:23:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:27 smithi179 ceph-mon[33312]: pgmap v1299: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:23:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:27 smithi167 ceph-mon[37701]: pgmap v1299: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:23:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:27 smithi167 ceph-mon[29496]: pgmap v1299: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:23:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:29 smithi179 ceph-mon[33312]: pgmap v1300: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:29.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:23:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:23:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:29 smithi167 ceph-mon[37701]: pgmap v1300: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:29 smithi167 ceph-mon[29496]: pgmap v1300: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:30.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:30 smithi179 ceph-mon[33312]: pgmap v1301: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:30 smithi167 ceph-mon[37701]: pgmap v1301: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:30 smithi167 ceph-mon[29496]: pgmap v1301: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:33.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:33 smithi179 ceph-mon[33312]: pgmap v1302: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:23:33.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:33 smithi167 ceph-mon[37701]: pgmap v1302: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:23:33.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:33 smithi167 ceph-mon[29496]: pgmap v1302: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:23:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:23:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:23:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:23:35.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:35 smithi179 ceph-mon[33312]: pgmap v1303: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:35.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:35 smithi167 ceph-mon[37701]: pgmap v1303: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:35 smithi167 ceph-mon[29496]: pgmap v1303: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:37.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:37 smithi179 ceph-mon[33312]: pgmap v1304: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:23:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:37 smithi167 ceph-mon[37701]: pgmap v1304: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:23:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:37 smithi167 ceph-mon[29496]: pgmap v1304: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:23:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:39 smithi179 ceph-mon[33312]: pgmap v1305: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:39.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:23:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:23:39.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:39 smithi167 ceph-mon[37701]: pgmap v1305: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:39.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:39 smithi167 ceph-mon[29496]: pgmap v1305: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:23:40.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:23:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:23:41.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:41 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:41 smithi179 ceph-mon[33312]: pgmap v1306: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:41 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:41 smithi167 ceph-mon[37701]: pgmap v1306: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:41 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:41 smithi167 ceph-mon[29496]: pgmap v1306: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:43 smithi179 ceph-mon[33312]: pgmap v1307: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:43 smithi167 ceph-mon[37701]: pgmap v1307: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:43 smithi167 ceph-mon[29496]: pgmap v1307: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:23:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:23:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:23:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:45 smithi179 ceph-mon[33312]: pgmap v1308: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:45.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:45 smithi167 ceph-mon[37701]: pgmap v1308: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:45 smithi167 ceph-mon[29496]: pgmap v1308: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:47 smithi179 ceph-mon[33312]: pgmap v1309: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:47 smithi167 ceph-mon[37701]: pgmap v1309: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:47.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:47 smithi167 ceph-mon[29496]: pgmap v1309: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:49 smithi179 ceph-mon[33312]: pgmap v1310: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:49.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:23:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:23:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:49 smithi167 ceph-mon[37701]: pgmap v1310: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:49 smithi167 ceph-mon[29496]: pgmap v1310: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:51 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:51 smithi179 ceph-mon[33312]: pgmap v1311: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:51 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:23:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:51 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:23:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:51 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:23:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:51 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:23:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:51 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:51 smithi167 ceph-mon[37701]: pgmap v1311: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:51 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:23:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:51 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:23:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:51 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:23:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:51 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:23:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:51 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:23:51.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:51 smithi167 ceph-mon[29496]: pgmap v1311: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:51.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:51 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:23:51.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:51 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:23:51.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:51 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:23:51.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:51 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:23:53.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:53 smithi179 ceph-mon[33312]: pgmap v1312: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:53 smithi167 ceph-mon[37701]: pgmap v1312: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:53.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:53 smithi167 ceph-mon[29496]: pgmap v1312: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:23:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:23:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:23:55.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:55 smithi179 ceph-mon[33312]: pgmap v1313: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:23:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:55 smithi167 ceph-mon[37701]: pgmap v1313: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:23:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:55 smithi167 ceph-mon[29496]: pgmap v1313: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:23:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:23:57.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:57 smithi179 ceph-mon[33312]: pgmap v1314: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:57 smithi167 ceph-mon[37701]: pgmap v1314: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:57 smithi167 ceph-mon[29496]: pgmap v1314: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:59.630 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:23:59 smithi179 ceph-mon[33312]: pgmap v1315: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:23:59 smithi167 ceph-mon[37701]: pgmap v1315: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:23:59 smithi167 ceph-mon[29496]: pgmap v1315: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:23:59.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:23:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:24:00.887 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:24:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=cleanup t=2024-06-18T13:24:00.580196188Z level=info msg="Completed cleanup jobs" duration=2.306286ms 2024-06-18T13:24:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:01 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:01 smithi179 ceph-mon[33312]: pgmap v1316: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:01.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:01 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:01 smithi167 ceph-mon[37701]: pgmap v1316: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:01 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:01 smithi167 ceph-mon[29496]: pgmap v1316: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:03 smithi179 ceph-mon[33312]: pgmap v1317: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:03.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:03 smithi167 ceph-mon[37701]: pgmap v1317: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:03 smithi167 ceph-mon[29496]: pgmap v1317: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:24:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:24:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:24:05.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:05 smithi179 ceph-mon[33312]: pgmap v1318: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:05 smithi167 ceph-mon[37701]: pgmap v1318: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:05 smithi167 ceph-mon[29496]: pgmap v1318: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:07 smithi179 ceph-mon[33312]: pgmap v1319: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:07 smithi167 ceph-mon[37701]: pgmap v1319: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:07 smithi167 ceph-mon[29496]: pgmap v1319: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:09 smithi179 ceph-mon[33312]: pgmap v1320: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:09.888 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:24:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:24:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:09 smithi167 ceph-mon[37701]: pgmap v1320: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:09 smithi167 ceph-mon[29496]: pgmap v1320: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:24:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:24:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:24:11.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:11 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:11 smithi179 ceph-mon[33312]: pgmap v1321: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:11 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:11 smithi167 ceph-mon[37701]: pgmap v1321: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:11 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:11 smithi167 ceph-mon[29496]: pgmap v1321: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:13 smithi179 ceph-mon[33312]: pgmap v1322: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:13 smithi167 ceph-mon[37701]: pgmap v1322: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:13 smithi167 ceph-mon[29496]: pgmap v1322: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:24:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:24:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:24:15.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:15 smithi179 ceph-mon[33312]: pgmap v1323: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:15 smithi167 ceph-mon[29496]: pgmap v1323: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:15 smithi167 ceph-mon[37701]: pgmap v1323: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:17 smithi179 ceph-mon[33312]: pgmap v1324: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:17 smithi167 ceph-mon[29496]: pgmap v1324: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:17 smithi167 ceph-mon[37701]: pgmap v1324: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:19.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:19 smithi179 ceph-mon[33312]: pgmap v1325: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:19.890 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:24:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:24:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:19 smithi167 ceph-mon[29496]: pgmap v1325: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:19 smithi167 ceph-mon[37701]: pgmap v1325: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:21 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:21 smithi179 ceph-mon[33312]: pgmap v1326: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:21 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:21 smithi167 ceph-mon[29496]: pgmap v1326: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:21 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:21 smithi167 ceph-mon[37701]: pgmap v1326: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:23 smithi179 ceph-mon[33312]: pgmap v1327: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:23 smithi167 ceph-mon[29496]: pgmap v1327: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:23 smithi167 ceph-mon[37701]: pgmap v1327: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:24:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:24:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:24:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:25 smithi179 ceph-mon[33312]: pgmap v1328: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:24:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:25 smithi167 ceph-mon[37701]: pgmap v1328: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:24:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:25 smithi167 ceph-mon[29496]: pgmap v1328: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:24:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:27 smithi179 ceph-mon[33312]: pgmap v1329: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:24:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:27 smithi167 ceph-mon[29496]: pgmap v1329: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:24:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:27 smithi167 ceph-mon[37701]: pgmap v1329: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:24:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:29 smithi179 ceph-mon[33312]: pgmap v1330: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:29.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:24:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:24:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:29 smithi167 ceph-mon[29496]: pgmap v1330: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:29 smithi167 ceph-mon[37701]: pgmap v1330: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:31.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:31 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:31 smithi179 ceph-mon[33312]: pgmap v1331: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:31 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:31 smithi167 ceph-mon[37701]: pgmap v1331: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:31 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:31 smithi167 ceph-mon[29496]: pgmap v1331: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:33 smithi179 ceph-mon[33312]: pgmap v1332: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:24:33.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:33 smithi167 ceph-mon[37701]: pgmap v1332: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:24:33.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:33 smithi167 ceph-mon[29496]: pgmap v1332: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:24:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:34 smithi179 ceph-mon[33312]: pgmap v1333: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:34 smithi167 ceph-mon[29496]: pgmap v1333: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:34 smithi167 ceph-mon[37701]: pgmap v1333: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:24:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:24:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:24:37.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:37 smithi179 ceph-mon[33312]: pgmap v1334: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:24:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:37 smithi167 ceph-mon[37701]: pgmap v1334: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:24:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:37 smithi167 ceph-mon[29496]: pgmap v1334: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:24:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:39 smithi179 ceph-mon[33312]: pgmap v1335: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:39.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:24:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:24:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:39 smithi167 ceph-mon[37701]: pgmap v1335: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:39.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:39 smithi167 ceph-mon[29496]: pgmap v1335: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:24:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:24:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:24:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:41 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:41 smithi179 ceph-mon[33312]: pgmap v1336: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:41 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:41 smithi167 ceph-mon[37701]: pgmap v1336: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:41 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:41 smithi167 ceph-mon[29496]: pgmap v1336: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:43 smithi179 ceph-mon[33312]: pgmap v1337: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:24:43.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:43 smithi167 ceph-mon[37701]: pgmap v1337: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:24:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:43 smithi167 ceph-mon[29496]: pgmap v1337: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:24:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:24:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:24:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:24:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:45 smithi179 ceph-mon[33312]: pgmap v1338: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:45 smithi167 ceph-mon[29496]: pgmap v1338: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:45.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:45 smithi167 ceph-mon[37701]: pgmap v1338: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:47 smithi179 ceph-mon[33312]: pgmap v1339: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:47 smithi167 ceph-mon[37701]: pgmap v1339: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:47.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:47 smithi167 ceph-mon[29496]: pgmap v1339: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:49 smithi179 ceph-mon[33312]: pgmap v1340: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:49.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:24:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:24:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:49 smithi167 ceph-mon[37701]: pgmap v1340: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:49 smithi167 ceph-mon[29496]: pgmap v1340: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:51 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:51 smithi179 ceph-mon[33312]: pgmap v1341: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:51 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:24:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:51 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:24:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:51 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:24:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:51 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:24:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:51 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:51 smithi167 ceph-mon[37701]: pgmap v1341: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:51 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:24:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:51 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:24:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:51 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:24:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:51 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:24:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:51 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:24:51.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:51 smithi167 ceph-mon[29496]: pgmap v1341: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:51.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:51 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:24:51.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:51 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:24:51.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:51 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:24:51.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:51 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:24:53.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:53 smithi179 ceph-mon[33312]: pgmap v1342: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:53 smithi167 ceph-mon[37701]: pgmap v1342: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:53.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:53 smithi167 ceph-mon[29496]: pgmap v1342: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:24:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:24:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:24:55.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:55 smithi179 ceph-mon[33312]: pgmap v1343: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:24:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:55 smithi167 ceph-mon[29496]: pgmap v1343: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:24:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:55 smithi167 ceph-mon[37701]: pgmap v1343: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:24:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:24:57.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:57 smithi179 ceph-mon[33312]: pgmap v1344: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:57 smithi167 ceph-mon[37701]: pgmap v1344: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:57 smithi167 ceph-mon[29496]: pgmap v1344: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:59.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:24:59 smithi179 ceph-mon[33312]: pgmap v1345: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:59.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:24:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:24:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:24:59 smithi167 ceph-mon[37701]: pgmap v1345: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:24:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:24:59 smithi167 ceph-mon[29496]: pgmap v1345: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:01 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:01 smithi179 ceph-mon[33312]: pgmap v1346: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:01 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:01 smithi167 ceph-mon[37701]: pgmap v1346: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:01 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:01 smithi167 ceph-mon[29496]: pgmap v1346: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:03 smithi179 ceph-mon[33312]: pgmap v1347: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:03.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:03 smithi167 ceph-mon[37701]: pgmap v1347: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:03 smithi167 ceph-mon[29496]: pgmap v1347: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:04.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:04 smithi179 ceph-mon[33312]: pgmap v1348: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:04 smithi167 ceph-mon[37701]: pgmap v1348: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:04 smithi167 ceph-mon[29496]: pgmap v1348: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:25:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:25:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:25:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:07 smithi179 ceph-mon[33312]: pgmap v1349: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:07.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:07 smithi167 ceph-mon[37701]: pgmap v1349: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:07 smithi167 ceph-mon[29496]: pgmap v1349: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:09 smithi179 ceph-mon[33312]: pgmap v1350: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:09.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:25:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:25:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:09 smithi167 ceph-mon[37701]: pgmap v1350: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:09 smithi167 ceph-mon[29496]: pgmap v1350: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:25:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:25:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:25:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:11 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:11 smithi179 ceph-mon[33312]: pgmap v1351: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:11 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:11 smithi167 ceph-mon[29496]: pgmap v1351: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:11 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:11 smithi167 ceph-mon[37701]: pgmap v1351: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:13 smithi179 ceph-mon[33312]: pgmap v1352: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:13 smithi167 ceph-mon[29496]: pgmap v1352: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:13 smithi167 ceph-mon[37701]: pgmap v1352: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:25:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:25:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:25:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:15 smithi179 ceph-mon[33312]: pgmap v1353: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:15 smithi167 ceph-mon[29496]: pgmap v1353: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:15 smithi167 ceph-mon[37701]: pgmap v1353: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:17 smithi179 ceph-mon[33312]: pgmap v1354: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:17 smithi167 ceph-mon[29496]: pgmap v1354: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:17 smithi167 ceph-mon[37701]: pgmap v1354: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:19.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:19 smithi179 ceph-mon[33312]: pgmap v1355: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:19.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:25:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:25:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:19 smithi167 ceph-mon[37701]: pgmap v1355: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:19 smithi167 ceph-mon[29496]: pgmap v1355: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:21.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:21 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:21 smithi179 ceph-mon[33312]: pgmap v1356: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:21 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:21 smithi167 ceph-mon[37701]: pgmap v1356: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:21 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:21 smithi167 ceph-mon[29496]: pgmap v1356: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:23 smithi179 ceph-mon[33312]: pgmap v1357: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:23 smithi167 ceph-mon[37701]: pgmap v1357: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:23 smithi167 ceph-mon[29496]: pgmap v1357: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:25:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:25:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:25:25.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:25 smithi179 ceph-mon[33312]: pgmap v1358: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:25:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:25 smithi167 ceph-mon[37701]: pgmap v1358: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:25:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:25 smithi167 ceph-mon[29496]: pgmap v1358: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:25:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:27 smithi179 ceph-mon[33312]: pgmap v1359: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:27 smithi167 ceph-mon[29496]: pgmap v1359: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:27 smithi167 ceph-mon[37701]: pgmap v1359: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:28 smithi179 ceph-mon[33312]: pgmap v1360: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:28 smithi167 ceph-mon[37701]: pgmap v1360: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:28 smithi167 ceph-mon[29496]: pgmap v1360: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:30.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:25:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:25:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:31 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:31 smithi179 ceph-mon[33312]: pgmap v1361: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:31 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:31 smithi167 ceph-mon[37701]: pgmap v1361: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:31 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:31 smithi167 ceph-mon[29496]: pgmap v1361: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:33.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:33 smithi179 ceph-mon[33312]: pgmap v1362: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:33.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:33 smithi167 ceph-mon[37701]: pgmap v1362: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:33.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:33 smithi167 ceph-mon[29496]: pgmap v1362: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:25:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:25:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:25:35.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:35 smithi179 ceph-mon[33312]: pgmap v1363: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:35.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:35 smithi167 ceph-mon[37701]: pgmap v1363: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:35 smithi167 ceph-mon[29496]: pgmap v1363: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:37.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:37 smithi179 ceph-mon[33312]: pgmap v1364: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:37 smithi167 ceph-mon[37701]: pgmap v1364: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:37 smithi167 ceph-mon[29496]: pgmap v1364: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:25:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:39 smithi179 ceph-mon[33312]: pgmap v1365: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:39.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:25:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:25:39.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:39 smithi167 ceph-mon[29496]: pgmap v1365: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:39 smithi167 ceph-mon[37701]: pgmap v1365: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:25:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:25:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:25:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:41 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:41 smithi179 ceph-mon[33312]: pgmap v1366: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:41 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:41 smithi167 ceph-mon[29496]: pgmap v1366: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:41 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:41 smithi167 ceph-mon[37701]: pgmap v1366: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:43 smithi179 ceph-mon[33312]: pgmap v1367: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:43 smithi167 ceph-mon[29496]: pgmap v1367: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:43 smithi167 ceph-mon[37701]: pgmap v1367: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:25:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:25:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:25:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:45 smithi179 ceph-mon[33312]: pgmap v1368: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:45.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:45 smithi167 ceph-mon[37701]: pgmap v1368: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:45 smithi167 ceph-mon[29496]: pgmap v1368: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:47.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:47 smithi179 ceph-mon[33312]: pgmap v1369: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:47 smithi167 ceph-mon[37701]: pgmap v1369: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:47.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:47 smithi167 ceph-mon[29496]: pgmap v1369: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:49 smithi179 ceph-mon[33312]: pgmap v1370: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:49.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:25:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:25:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:49 smithi167 ceph-mon[37701]: pgmap v1370: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:49 smithi167 ceph-mon[29496]: pgmap v1370: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:50.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:50 smithi179 ceph-mon[33312]: pgmap v1371: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:50 smithi167 ceph-mon[29496]: pgmap v1371: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:25:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:50 smithi167 ceph-mon[37701]: pgmap v1371: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:51 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:25:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:51 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:25:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:51 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:25:52.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:52 smithi179 ceph-mon[33312]: pgmap v1372: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:52.976 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:52 smithi167 ceph-mon[29496]: pgmap v1372: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:52 smithi167 ceph-mon[37701]: pgmap v1372: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:55.336 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:25:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:25:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:25:55.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:55.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:55 smithi179 ceph-mon[33312]: pgmap v1373: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:55.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:55 smithi167 ceph-mon[37701]: pgmap v1373: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:55.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:55 smithi167 ceph-mon[29496]: pgmap v1373: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:25:56.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:25:56.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:56.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:56.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:25:56.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:25:56.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:25:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:25:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:25:56.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:56.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:25:56.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:56.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:56.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:25:56.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:25:56.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:25:57.636 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:57 smithi179 ceph-mon[33312]: pgmap v1374: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:57.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:57 smithi167 ceph-mon[37701]: pgmap v1374: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:57.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:57 smithi167 ceph-mon[29496]: pgmap v1374: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:59.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:25:59 smithi179 ceph-mon[33312]: pgmap v1375: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:59.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:25:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:25:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:25:59 smithi167 ceph-mon[37701]: pgmap v1375: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:25:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:25:59 smithi167 ceph-mon[29496]: pgmap v1375: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:01 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:01 smithi179 ceph-mon[33312]: pgmap v1376: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:01 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:01 smithi167 ceph-mon[37701]: pgmap v1376: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:01 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:01 smithi167 ceph-mon[29496]: pgmap v1376: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:03 smithi179 ceph-mon[33312]: pgmap v1377: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:03 smithi167 ceph-mon[37701]: pgmap v1377: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:03 smithi167 ceph-mon[29496]: pgmap v1377: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:26:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:26:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:26:05.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:05 smithi179 ceph-mon[33312]: pgmap v1378: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:05 smithi167 ceph-mon[37701]: pgmap v1378: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:05 smithi167 ceph-mon[29496]: pgmap v1378: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:07 smithi179 ceph-mon[33312]: pgmap v1379: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:07 smithi167 ceph-mon[37701]: pgmap v1379: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:07 smithi167 ceph-mon[29496]: pgmap v1379: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:09 smithi179 ceph-mon[33312]: pgmap v1380: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:09.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:26:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:26:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:09 smithi167 ceph-mon[37701]: pgmap v1380: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:09 smithi167 ceph-mon[29496]: pgmap v1380: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:26:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:26:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:26:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:11 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:11 smithi179 ceph-mon[33312]: pgmap v1381: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:11 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:11 smithi167 ceph-mon[37701]: pgmap v1381: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:11 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:11 smithi167 ceph-mon[29496]: pgmap v1381: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:13 smithi179 ceph-mon[33312]: pgmap v1382: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:13 smithi167 ceph-mon[37701]: pgmap v1382: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:13 smithi167 ceph-mon[29496]: pgmap v1382: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:14 smithi167 ceph-mon[37701]: pgmap v1383: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:14 smithi167 ceph-mon[29496]: pgmap v1383: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:15.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:14 smithi179 ceph-mon[33312]: pgmap v1383: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:26:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:26:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:26:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:17 smithi179 ceph-mon[33312]: pgmap v1384: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:17 smithi167 ceph-mon[37701]: pgmap v1384: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:17 smithi167 ceph-mon[29496]: pgmap v1384: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:19.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:19 smithi179 ceph-mon[33312]: pgmap v1385: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:19.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:26:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:26:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:19 smithi167 ceph-mon[37701]: pgmap v1385: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:19 smithi167 ceph-mon[29496]: pgmap v1385: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:21 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:21.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:21 smithi179 ceph-mon[33312]: pgmap v1386: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:21 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:21 smithi167 ceph-mon[37701]: pgmap v1386: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:21 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:21 smithi167 ceph-mon[29496]: pgmap v1386: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:23.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:23 smithi179 ceph-mon[33312]: pgmap v1387: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:23 smithi167 ceph-mon[29496]: pgmap v1387: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:23 smithi167 ceph-mon[37701]: pgmap v1387: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:26:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:26:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:26:25.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:25 smithi179 ceph-mon[33312]: pgmap v1388: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:26:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:25 smithi167 ceph-mon[37701]: pgmap v1388: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:26:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:25 smithi167 ceph-mon[29496]: pgmap v1388: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:26:27.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:27 smithi179 ceph-mon[33312]: pgmap v1389: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:27 smithi167 ceph-mon[29496]: pgmap v1389: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:27 smithi167 ceph-mon[37701]: pgmap v1389: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:29.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:26:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:26:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:29 smithi179 ceph-mon[33312]: pgmap v1390: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:29 smithi167 ceph-mon[37701]: pgmap v1390: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:29 smithi167 ceph-mon[29496]: pgmap v1390: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:31 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:31 smithi179 ceph-mon[33312]: pgmap v1391: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:31 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:31 smithi167 ceph-mon[37701]: pgmap v1391: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:31 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:31 smithi167 ceph-mon[29496]: pgmap v1391: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:32.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:32 smithi179 ceph-mon[33312]: pgmap v1392: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:32 smithi167 ceph-mon[37701]: pgmap v1392: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:32 smithi167 ceph-mon[29496]: pgmap v1392: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:26:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:26:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:26:35.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:35 smithi179 ceph-mon[33312]: pgmap v1393: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:35.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:35 smithi167 ceph-mon[37701]: pgmap v1393: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:35 smithi167 ceph-mon[29496]: pgmap v1393: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:37.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:37 smithi179 ceph-mon[33312]: pgmap v1394: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:37 smithi167 ceph-mon[37701]: pgmap v1394: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:37 smithi167 ceph-mon[29496]: pgmap v1394: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:39.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:39 smithi179 ceph-mon[33312]: pgmap v1395: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:39.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:26:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:26:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:39 smithi167 ceph-mon[37701]: pgmap v1395: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:39.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:39 smithi167 ceph-mon[29496]: pgmap v1395: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:26:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:26:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:26:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:41 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:41 smithi179 ceph-mon[33312]: pgmap v1396: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:41 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:41 smithi167 ceph-mon[37701]: pgmap v1396: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:41 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:41 smithi167 ceph-mon[29496]: pgmap v1396: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:43 smithi179 ceph-mon[33312]: pgmap v1397: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:43 smithi167 ceph-mon[37701]: pgmap v1397: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:43 smithi167 ceph-mon[29496]: pgmap v1397: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:44 smithi179 ceph-mon[33312]: pgmap v1398: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:44 smithi167 ceph-mon[29496]: pgmap v1398: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:44 smithi167 ceph-mon[37701]: pgmap v1398: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:26:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:26:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:26:47.889 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:47 smithi179 ceph-mon[33312]: pgmap v1399: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:47 smithi167 ceph-mon[37701]: pgmap v1399: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:47.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:47 smithi167 ceph-mon[29496]: pgmap v1399: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:49.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:49 smithi179 ceph-mon[33312]: pgmap v1400: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:49.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:26:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:26:49.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:49 smithi167 ceph-mon[37701]: pgmap v1400: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:49.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:49 smithi167 ceph-mon[29496]: pgmap v1400: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:51.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:51 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:51 smithi179 ceph-mon[33312]: pgmap v1401: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:51 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:51 smithi167 ceph-mon[37701]: pgmap v1401: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:51 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:26:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:51 smithi167 ceph-mon[29496]: pgmap v1401: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:53.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:53 smithi179 ceph-mon[33312]: pgmap v1402: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:53.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:53 smithi167 ceph-mon[37701]: pgmap v1402: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:53.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:53 smithi167 ceph-mon[29496]: pgmap v1402: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:26:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:26:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:26:55.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:55 smithi179 ceph-mon[33312]: pgmap v1403: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:55.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:26:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:55 smithi167 ceph-mon[29496]: pgmap v1403: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:26:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:55 smithi167 ceph-mon[37701]: pgmap v1403: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:26:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:26:56.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:26:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:26:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:26:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:26:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:26:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:26:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:26:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:26:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:26:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:26:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:26:56.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:26:57.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:57 smithi179 ceph-mon[33312]: pgmap v1404: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:57 smithi167 ceph-mon[37701]: pgmap v1404: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:57 smithi167 ceph-mon[29496]: pgmap v1404: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:26:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:26:58 smithi179 ceph-mon[33312]: pgmap v1405: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:26:58 smithi167 ceph-mon[37701]: pgmap v1405: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:26:58.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:26:58 smithi167 ceph-mon[29496]: pgmap v1405: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:00.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:26:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:27:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:01 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:01 smithi179 ceph-mon[33312]: pgmap v1406: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:01.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:01 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:01 smithi167 ceph-mon[37701]: pgmap v1406: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:01 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:01 smithi167 ceph-mon[29496]: pgmap v1406: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:03.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:03 smithi179 ceph-mon[33312]: pgmap v1407: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:03 smithi167 ceph-mon[37701]: pgmap v1407: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:03 smithi167 ceph-mon[29496]: pgmap v1407: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:27:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:27:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:27:05.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:05 smithi179 ceph-mon[33312]: pgmap v1408: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:05.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:05 smithi167 ceph-mon[37701]: pgmap v1408: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:05.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:05 smithi167 ceph-mon[29496]: pgmap v1408: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:06 smithi167 ceph-mon[37701]: pgmap v1409: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:06 smithi167 ceph-mon[29496]: pgmap v1409: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:06 smithi179 ceph-mon[33312]: pgmap v1409: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:09.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:09 smithi179 ceph-mon[33312]: pgmap v1410: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:09.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:27:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:27:09.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:09 smithi167 ceph-mon[37701]: pgmap v1410: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:09.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:09 smithi167 ceph-mon[29496]: pgmap v1410: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:27:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:27:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:27:11.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:11 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:11 smithi179 ceph-mon[33312]: pgmap v1411: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:11.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:11 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:11 smithi167 ceph-mon[37701]: pgmap v1411: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:11 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:11 smithi167 ceph-mon[29496]: pgmap v1411: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:13 smithi179 ceph-mon[33312]: pgmap v1412: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:27:13.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:13 smithi167 ceph-mon[37701]: pgmap v1412: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:27:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:13 smithi167 ceph-mon[29496]: pgmap v1412: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:27:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:14 smithi179 ceph-mon[33312]: pgmap v1413: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:14 smithi167 ceph-mon[37701]: pgmap v1413: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:14 smithi167 ceph-mon[29496]: pgmap v1413: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:27:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:27:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:27:17.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:17 smithi179 ceph-mon[33312]: pgmap v1414: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:27:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:17 smithi167 ceph-mon[37701]: pgmap v1414: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:27:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:17 smithi167 ceph-mon[29496]: pgmap v1414: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:27:19.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:19 smithi179 ceph-mon[33312]: pgmap v1415: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:19.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:27:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:27:19.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:19 smithi167 ceph-mon[37701]: pgmap v1415: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:19.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:19 smithi167 ceph-mon[29496]: pgmap v1415: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:21 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:21 smithi179 ceph-mon[33312]: pgmap v1416: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:21 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:21 smithi167 ceph-mon[37701]: pgmap v1416: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:21 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:21 smithi167 ceph-mon[29496]: pgmap v1416: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:23.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:23 smithi179 ceph-mon[33312]: pgmap v1417: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:23 smithi167 ceph-mon[37701]: pgmap v1417: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:23 smithi167 ceph-mon[29496]: pgmap v1417: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:24 smithi167 ceph-mon[37701]: pgmap v1418: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:24 smithi167 ceph-mon[29496]: pgmap v1418: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:24 smithi179 ceph-mon[33312]: pgmap v1418: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:27:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:27:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:27:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:27:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:27:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:27:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:26 smithi167 ceph-mon[29496]: pgmap v1419: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:26 smithi167 ceph-mon[37701]: pgmap v1419: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:27.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:26 smithi179 ceph-mon[33312]: pgmap v1419: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:29.851 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:29 smithi179 ceph-mon[33312]: pgmap v1420: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:29.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:29 smithi167 ceph-mon[29496]: pgmap v1420: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:29 smithi167 ceph-mon[37701]: pgmap v1420: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:30.136 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:27:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:27:31.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:31 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:31 smithi179 ceph-mon[33312]: pgmap v1421: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:31 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:31 smithi167 ceph-mon[37701]: pgmap v1421: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:31 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:31 smithi167 ceph-mon[29496]: pgmap v1421: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:32 smithi167 ceph-mon[37701]: pgmap v1422: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:32 smithi167 ceph-mon[29496]: pgmap v1422: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:33.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:32 smithi179 ceph-mon[33312]: pgmap v1422: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:35.476 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:27:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:27:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:27:35.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:35 smithi179 ceph-mon[33312]: pgmap v1423: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:35.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:35 smithi167 ceph-mon[37701]: pgmap v1423: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:35.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:35 smithi167 ceph-mon[29496]: pgmap v1423: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:37.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:37 smithi179 ceph-mon[33312]: pgmap v1424: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:37.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:37 smithi167 ceph-mon[37701]: pgmap v1424: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:37.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:37 smithi167 ceph-mon[29496]: pgmap v1424: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:38.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:38 smithi167 ceph-mon[37701]: pgmap v1425: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:38 smithi167 ceph-mon[29496]: pgmap v1425: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:39.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:38 smithi179 ceph-mon[33312]: pgmap v1425: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:40.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:27:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:27:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:27:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:27:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:27:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:41 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:41 smithi179 ceph-mon[33312]: pgmap v1426: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:41 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:41 smithi167 ceph-mon[37701]: pgmap v1426: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:41 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:41 smithi167 ceph-mon[29496]: pgmap v1426: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:43 smithi179 ceph-mon[33312]: pgmap v1427: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:43 smithi167 ceph-mon[37701]: pgmap v1427: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:43 smithi167 ceph-mon[29496]: pgmap v1427: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:44 smithi179 ceph-mon[33312]: pgmap v1428: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:44 smithi167 ceph-mon[37701]: pgmap v1428: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:44 smithi167 ceph-mon[29496]: pgmap v1428: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:27:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:27:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:27:47.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:47 smithi179 ceph-mon[33312]: pgmap v1429: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:27:47.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:47 smithi167 ceph-mon[37701]: pgmap v1429: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:27:47.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:47 smithi167 ceph-mon[29496]: pgmap v1429: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:27:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:48 smithi167 ceph-mon[37701]: pgmap v1430: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:48 smithi167 ceph-mon[29496]: pgmap v1430: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:49.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:48 smithi179 ceph-mon[33312]: pgmap v1430: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:50.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:27:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:27:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:51 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:51 smithi179 ceph-mon[33312]: pgmap v1431: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:51 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:51 smithi167 ceph-mon[37701]: pgmap v1431: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:51 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:27:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:51 smithi167 ceph-mon[29496]: pgmap v1431: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:52.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:52 smithi179 ceph-mon[33312]: pgmap v1432: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:27:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:52 smithi167 ceph-mon[37701]: pgmap v1432: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:27:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:52 smithi167 ceph-mon[29496]: pgmap v1432: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:27:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:54 smithi167 ceph-mon[29496]: pgmap v1433: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:54 smithi167 ceph-mon[37701]: pgmap v1433: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:55.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:54 smithi179 ceph-mon[33312]: pgmap v1433: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:27:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:27:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:27:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:27:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:27:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:27:56.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:27:56.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:56 smithi167 ceph-mon[37701]: pgmap v1434: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:27:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:56 smithi167 ceph-mon[29496]: pgmap v1434: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:27:57.037 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:56 smithi179 ceph-mon[33312]: pgmap v1434: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:57.037 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:27:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:57 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:27:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:57 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:27:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:57 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:27:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:57 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:27:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:57 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:27:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:57 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:27:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:57 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:27:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:57 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:27:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:57 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:27:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:27:58 smithi167 ceph-mon[37701]: pgmap v1435: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:58.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:27:58 smithi167 ceph-mon[29496]: pgmap v1435: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:27:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:27:58 smithi179 ceph-mon[33312]: pgmap v1435: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:00.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:27:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:28:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:00 smithi167 ceph-mon[37701]: pgmap v1436: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:00 smithi167 ceph-mon[29496]: pgmap v1436: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:00 smithi179 ceph-mon[33312]: pgmap v1436: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:02 smithi167 ceph-mon[37701]: pgmap v1437: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:02 smithi167 ceph-mon[29496]: pgmap v1437: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:03.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:02 smithi179 ceph-mon[33312]: pgmap v1437: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:04 smithi167 ceph-mon[37701]: pgmap v1438: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:04 smithi167 ceph-mon[29496]: pgmap v1438: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:05.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:04 smithi179 ceph-mon[33312]: pgmap v1438: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:28:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:28:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:28:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:06 smithi167 ceph-mon[37701]: pgmap v1439: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:06 smithi167 ceph-mon[29496]: pgmap v1439: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:06 smithi179 ceph-mon[33312]: pgmap v1439: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:08 smithi167 ceph-mon[29496]: pgmap v1440: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:08 smithi167 ceph-mon[37701]: pgmap v1440: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:09.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:08 smithi179 ceph-mon[33312]: pgmap v1440: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:10.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:28:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:28:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:28:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:28:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:28:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:11 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:11 smithi179 ceph-mon[33312]: pgmap v1441: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:11 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:11 smithi167 ceph-mon[29496]: pgmap v1441: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:11 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:11 smithi167 ceph-mon[37701]: pgmap v1441: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:13.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:13 smithi179 ceph-mon[33312]: pgmap v1442: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:13.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:13 smithi167 ceph-mon[37701]: pgmap v1442: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:13.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:13 smithi167 ceph-mon[29496]: pgmap v1442: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:28:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:28:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:28:15.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:15 smithi179 ceph-mon[33312]: pgmap v1443: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:15.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:15 smithi167 ceph-mon[29496]: pgmap v1443: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:15.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:15 smithi167 ceph-mon[37701]: pgmap v1443: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:17 smithi179 ceph-mon[33312]: pgmap v1444: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:17 smithi167 ceph-mon[29496]: pgmap v1444: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:17 smithi167 ceph-mon[37701]: pgmap v1444: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:18 smithi167 ceph-mon[37701]: pgmap v1445: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:18 smithi167 ceph-mon[29496]: pgmap v1445: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:19.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:18 smithi179 ceph-mon[33312]: pgmap v1445: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:20.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:28:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:28:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:20 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:20 smithi167 ceph-mon[37701]: pgmap v1446: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:20 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:20 smithi167 ceph-mon[29496]: pgmap v1446: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:21.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:20 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:21.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:20 smithi179 ceph-mon[33312]: pgmap v1446: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:22 smithi167 ceph-mon[37701]: pgmap v1447: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:22 smithi167 ceph-mon[29496]: pgmap v1447: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:23.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:22 smithi179 ceph-mon[33312]: pgmap v1447: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:24 smithi167 ceph-mon[37701]: pgmap v1448: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:24 smithi167 ceph-mon[29496]: pgmap v1448: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:24 smithi179 ceph-mon[33312]: pgmap v1448: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:28:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:28:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:28:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:28:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:28:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:28:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:26 smithi167 ceph-mon[37701]: pgmap v1449: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:26 smithi167 ceph-mon[29496]: pgmap v1449: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:27.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:26 smithi179 ceph-mon[33312]: pgmap v1449: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:28 smithi167 ceph-mon[37701]: pgmap v1450: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:28 smithi167 ceph-mon[29496]: pgmap v1450: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:28 smithi179 ceph-mon[33312]: pgmap v1450: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:30.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:28:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:28:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:30 smithi167 ceph-mon[37701]: pgmap v1451: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:30 smithi167 ceph-mon[29496]: pgmap v1451: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:31.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:30 smithi179 ceph-mon[33312]: pgmap v1451: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:32 smithi167 ceph-mon[37701]: pgmap v1452: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:32 smithi167 ceph-mon[29496]: pgmap v1452: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:32 smithi179 ceph-mon[33312]: pgmap v1452: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:34 smithi167 ceph-mon[37701]: pgmap v1453: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:34 smithi167 ceph-mon[29496]: pgmap v1453: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:35.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:34 smithi179 ceph-mon[33312]: pgmap v1453: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:28:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:28:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:28:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:36 smithi167 ceph-mon[37701]: pgmap v1454: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:36 smithi167 ceph-mon[29496]: pgmap v1454: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:36 smithi179 ceph-mon[33312]: pgmap v1454: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:38.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:38 smithi167 ceph-mon[37701]: pgmap v1455: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:38.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:38 smithi167 ceph-mon[29496]: pgmap v1455: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:38 smithi179 ceph-mon[33312]: pgmap v1455: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:40.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:28:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:28:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:40 smithi179 ceph-mon[33312]: pgmap v1456: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:28:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:40 smithi167 ceph-mon[37701]: pgmap v1456: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:28:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:40 smithi167 ceph-mon[29496]: pgmap v1456: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:28:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:42 smithi167 ceph-mon[29496]: pgmap v1457: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:42 smithi167 ceph-mon[37701]: pgmap v1457: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:43.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:42 smithi179 ceph-mon[33312]: pgmap v1457: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:44 smithi167 ceph-mon[29496]: pgmap v1458: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:44 smithi167 ceph-mon[37701]: pgmap v1458: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:44 smithi179 ceph-mon[33312]: pgmap v1458: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:28:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:28:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:28:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:46 smithi167 ceph-mon[37701]: pgmap v1459: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:46 smithi167 ceph-mon[29496]: pgmap v1459: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:46 smithi179 ceph-mon[33312]: pgmap v1459: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:48 smithi167 ceph-mon[37701]: pgmap v1460: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:48 smithi167 ceph-mon[29496]: pgmap v1460: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:49.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:48 smithi179 ceph-mon[33312]: pgmap v1460: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:50.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:28:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:28:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:50 smithi167 ceph-mon[37701]: pgmap v1461: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:50 smithi167 ceph-mon[29496]: pgmap v1461: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:28:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:50 smithi179 ceph-mon[33312]: pgmap v1461: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:52 smithi167 ceph-mon[29496]: pgmap v1462: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:52.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:52 smithi167 ceph-mon[37701]: pgmap v1462: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:53.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:52 smithi179 ceph-mon[33312]: pgmap v1462: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:54 smithi167 ceph-mon[37701]: pgmap v1463: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:54 smithi167 ceph-mon[29496]: pgmap v1463: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:54 smithi179 ceph-mon[33312]: pgmap v1463: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:28:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:28:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:28:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:28:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:28:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:28:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:28:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:56 smithi167 ceph-mon[37701]: pgmap v1464: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:56 smithi167 ceph-mon[29496]: pgmap v1464: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:57.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:56 smithi179 ceph-mon[33312]: pgmap v1464: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:28:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:57 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:28:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:57 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:57 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:57 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:57.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:57 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:57 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:28:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:57 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:57.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:57 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:57.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:57 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:57.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:57 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:57 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:28:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:57 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:57 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:57 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:57 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:58 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:28:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:58 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:28:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:58 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:28:58 smithi167 ceph-mon[37701]: pgmap v1465: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:58.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:58 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:28:58.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:58 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:28:58.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:58 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:58.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:28:58 smithi167 ceph-mon[29496]: pgmap v1465: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:28:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:58 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:28:59.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:58 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:28:59.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:58 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:28:59.138 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:28:58 smithi179 ceph-mon[33312]: pgmap v1465: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:00.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:28:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:29:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:00 smithi167 ceph-mon[37701]: pgmap v1466: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:00 smithi167 ceph-mon[29496]: pgmap v1466: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:00 smithi179 ceph-mon[33312]: pgmap v1466: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:02 smithi167 ceph-mon[37701]: pgmap v1467: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:02 smithi167 ceph-mon[29496]: pgmap v1467: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:03.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:02 smithi179 ceph-mon[33312]: pgmap v1467: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:04 smithi167 ceph-mon[37701]: pgmap v1468: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:04 smithi167 ceph-mon[29496]: pgmap v1468: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:05.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:04 smithi179 ceph-mon[33312]: pgmap v1468: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:29:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:29:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:29:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:06 smithi167 ceph-mon[37701]: pgmap v1469: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:06 smithi167 ceph-mon[29496]: pgmap v1469: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:07.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:06 smithi179 ceph-mon[33312]: pgmap v1469: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:08 smithi167 ceph-mon[37701]: pgmap v1470: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:08 smithi167 ceph-mon[29496]: pgmap v1470: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:09.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:08 smithi179 ceph-mon[33312]: pgmap v1470: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:10.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:29:09 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:29:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:10 smithi167 ceph-mon[29496]: pgmap v1471: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:29:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:10 smithi167 ceph-mon[37701]: pgmap v1471: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:10.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:29:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:10 smithi179 ceph-mon[33312]: pgmap v1471: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:29:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:12 smithi167 ceph-mon[37701]: pgmap v1472: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:29:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:12 smithi167 ceph-mon[29496]: pgmap v1472: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:29:13.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:12 smithi179 ceph-mon[33312]: pgmap v1472: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:29:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:14 smithi167 ceph-mon[37701]: pgmap v1473: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:14 smithi167 ceph-mon[29496]: pgmap v1473: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:15.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:14 smithi179 ceph-mon[33312]: pgmap v1473: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:29:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:29:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:29:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:16 smithi167 ceph-mon[37701]: pgmap v1474: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:16 smithi167 ceph-mon[29496]: pgmap v1474: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:17.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:16 smithi179 ceph-mon[33312]: pgmap v1474: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:18 smithi167 ceph-mon[37701]: pgmap v1475: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:18 smithi167 ceph-mon[29496]: pgmap v1475: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:18 smithi179 ceph-mon[33312]: pgmap v1475: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:20.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:29:19 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:29:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:20 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:20 smithi167 ceph-mon[37701]: pgmap v1476: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:20 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:20 smithi167 ceph-mon[29496]: pgmap v1476: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:20 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:20 smithi179 ceph-mon[33312]: pgmap v1476: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:22 smithi167 ceph-mon[37701]: pgmap v1477: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:29:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:22 smithi167 ceph-mon[29496]: pgmap v1477: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:29:23.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:22 smithi179 ceph-mon[33312]: pgmap v1477: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:29:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:24 smithi167 ceph-mon[29496]: pgmap v1478: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:24 smithi167 ceph-mon[37701]: pgmap v1478: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:24 smithi179 ceph-mon[33312]: pgmap v1478: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:29:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:29:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:29:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:29:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:29:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:29:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:26 smithi167 ceph-mon[37701]: pgmap v1479: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:26 smithi167 ceph-mon[29496]: pgmap v1479: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:27.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:26 smithi179 ceph-mon[33312]: pgmap v1479: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:28 smithi167 ceph-mon[37701]: pgmap v1480: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:28 smithi167 ceph-mon[29496]: pgmap v1480: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:28 smithi179 ceph-mon[33312]: pgmap v1480: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:30.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:29:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:29:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:30 smithi167 ceph-mon[37701]: pgmap v1481: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:30 smithi167 ceph-mon[29496]: pgmap v1481: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:31.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:30 smithi179 ceph-mon[33312]: pgmap v1481: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:32 smithi167 ceph-mon[37701]: pgmap v1482: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:32 smithi167 ceph-mon[29496]: pgmap v1482: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:32 smithi179 ceph-mon[33312]: pgmap v1482: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:34.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:34 smithi167 ceph-mon[37701]: pgmap v1483: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:34 smithi167 ceph-mon[29496]: pgmap v1483: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:34 smithi179 ceph-mon[33312]: pgmap v1483: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:29:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:29:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:29:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:36 smithi167 ceph-mon[37701]: pgmap v1484: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:36 smithi167 ceph-mon[29496]: pgmap v1484: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:36 smithi179 ceph-mon[33312]: pgmap v1484: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:38.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:38 smithi167 ceph-mon[37701]: pgmap v1485: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:38 smithi167 ceph-mon[29496]: pgmap v1485: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:39.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:38 smithi179 ceph-mon[33312]: pgmap v1485: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:40.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:29:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:29:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:40 smithi167 ceph-mon[37701]: pgmap v1486: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:29:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:40 smithi167 ceph-mon[29496]: pgmap v1486: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:29:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:40 smithi179 ceph-mon[33312]: pgmap v1486: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:29:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:42 smithi167 ceph-mon[29496]: pgmap v1487: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:42 smithi167 ceph-mon[37701]: pgmap v1487: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:42 smithi179 ceph-mon[33312]: pgmap v1487: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:44 smithi167 ceph-mon[37701]: pgmap v1488: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:44 smithi167 ceph-mon[29496]: pgmap v1488: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:45.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:44 smithi179 ceph-mon[33312]: pgmap v1488: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:29:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:29:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:29:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:46 smithi167 ceph-mon[37701]: pgmap v1489: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:46 smithi167 ceph-mon[29496]: pgmap v1489: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:46 smithi179 ceph-mon[33312]: pgmap v1489: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:48 smithi167 ceph-mon[37701]: pgmap v1490: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:48 smithi167 ceph-mon[29496]: pgmap v1490: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:49.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:48 smithi179 ceph-mon[33312]: pgmap v1490: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:50.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:29:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:29:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:50 smithi167 ceph-mon[37701]: pgmap v1491: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:50 smithi167 ceph-mon[29496]: pgmap v1491: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:29:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:50 smithi179 ceph-mon[33312]: pgmap v1491: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:52 smithi167 ceph-mon[29496]: pgmap v1492: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:29:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:52 smithi167 ceph-mon[37701]: pgmap v1492: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:29:53.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:52 smithi179 ceph-mon[33312]: pgmap v1492: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:29:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:54 smithi167 ceph-mon[29496]: pgmap v1493: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:54.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:54 smithi167 ceph-mon[37701]: pgmap v1493: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:54 smithi179 ceph-mon[33312]: pgmap v1493: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:29:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:29:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:29:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:29:55.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:29:55.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:29:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:29:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:56 smithi167 ceph-mon[37701]: pgmap v1494: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:29:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:56 smithi167 ceph-mon[29496]: pgmap v1494: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:29:57.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:56 smithi179 ceph-mon[33312]: pgmap v1494: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:29:58.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:57 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:29:58.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:57 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:29:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:57 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:29:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:58 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:29:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:58 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:29:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:58 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:29:59.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:29:58 smithi167 ceph-mon[37701]: pgmap v1495: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:59.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:58 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:29:59.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:58 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:29:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:58 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:29:59.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:29:58 smithi167 ceph-mon[29496]: pgmap v1495: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:29:59.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:58 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:29:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:58 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:29:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:58 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:29:59.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:29:58 smithi179 ceph-mon[33312]: pgmap v1495: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:00.264 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:30:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:30:00.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:00 smithi179 ceph-mon[33312]: overall HEALTH_WARN 2 pool(s) do not have an application enabled 2024-06-18T13:30:00.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:00 smithi167 ceph-mon[37701]: overall HEALTH_WARN 2 pool(s) do not have an application enabled 2024-06-18T13:30:00.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:00 smithi167 ceph-mon[29496]: overall HEALTH_WARN 2 pool(s) do not have an application enabled 2024-06-18T13:30:01.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:01 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:01.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:01 smithi179 ceph-mon[33312]: pgmap v1496: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:01 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:01 smithi167 ceph-mon[37701]: pgmap v1496: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:01 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:01.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:01 smithi167 ceph-mon[29496]: pgmap v1496: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:02 smithi167 ceph-mon[37701]: pgmap v1497: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:30:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:02 smithi167 ceph-mon[29496]: pgmap v1497: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:30:03.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:02 smithi179 ceph-mon[33312]: pgmap v1497: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:30:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:04 smithi167 ceph-mon[37701]: pgmap v1498: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:04 smithi167 ceph-mon[29496]: pgmap v1498: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:05.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:04 smithi179 ceph-mon[33312]: pgmap v1498: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:30:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:30:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:30:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:06 smithi167 ceph-mon[37701]: pgmap v1499: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:30:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:06 smithi167 ceph-mon[29496]: pgmap v1499: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:30:07.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:06 smithi179 ceph-mon[33312]: pgmap v1499: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:30:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:08 smithi167 ceph-mon[37701]: pgmap v1500: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:08 smithi167 ceph-mon[29496]: pgmap v1500: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:09.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:08 smithi179 ceph-mon[33312]: pgmap v1500: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:10.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:30:10 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:30:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:10 smithi167 ceph-mon[37701]: pgmap v1501: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:30:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:10 smithi167 ceph-mon[29496]: pgmap v1501: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:30:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:10 smithi179 ceph-mon[33312]: pgmap v1501: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:30:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:12 smithi167 ceph-mon[37701]: pgmap v1502: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:12 smithi167 ceph-mon[29496]: pgmap v1502: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:13.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:12 smithi179 ceph-mon[33312]: pgmap v1502: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:14 smithi167 ceph-mon[37701]: pgmap v1503: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:14 smithi167 ceph-mon[29496]: pgmap v1503: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:15.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:14 smithi179 ceph-mon[33312]: pgmap v1503: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:30:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:30:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:30:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:16 smithi167 ceph-mon[37701]: pgmap v1504: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:16 smithi167 ceph-mon[29496]: pgmap v1504: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:17.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:16 smithi179 ceph-mon[33312]: pgmap v1504: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:18 smithi167 ceph-mon[37701]: pgmap v1505: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:18 smithi167 ceph-mon[29496]: pgmap v1505: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:19.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:18 smithi179 ceph-mon[33312]: pgmap v1505: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:20.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:30:20 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:30:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:20 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:20 smithi167 ceph-mon[37701]: pgmap v1506: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:20 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:20 smithi167 ceph-mon[29496]: pgmap v1506: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:20 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:20 smithi179 ceph-mon[33312]: pgmap v1506: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:22 smithi167 ceph-mon[29496]: pgmap v1507: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:22 smithi167 ceph-mon[37701]: pgmap v1507: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:23.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:22 smithi179 ceph-mon[33312]: pgmap v1507: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:24 smithi167 ceph-mon[29496]: pgmap v1508: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:24 smithi167 ceph-mon[37701]: pgmap v1508: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:25.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:24 smithi179 ceph-mon[33312]: pgmap v1508: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:30:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:30:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:30:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:30:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:30:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:30:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:26 smithi167 ceph-mon[29496]: pgmap v1509: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:26 smithi167 ceph-mon[37701]: pgmap v1509: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:27.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:26 smithi179 ceph-mon[33312]: pgmap v1509: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:28 smithi167 ceph-mon[37701]: pgmap v1510: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:28 smithi167 ceph-mon[29496]: pgmap v1510: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:28 smithi179 ceph-mon[33312]: pgmap v1510: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:30.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:30:30 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:30:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:30 smithi167 ceph-mon[37701]: pgmap v1511: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:30 smithi167 ceph-mon[29496]: pgmap v1511: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:30 smithi179 ceph-mon[33312]: pgmap v1511: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:32 smithi167 ceph-mon[37701]: pgmap v1512: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:32 smithi167 ceph-mon[29496]: pgmap v1512: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:32 smithi179 ceph-mon[33312]: pgmap v1512: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:34 smithi167 ceph-mon[37701]: pgmap v1513: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:34 smithi167 ceph-mon[29496]: pgmap v1513: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:34 smithi179 ceph-mon[33312]: pgmap v1513: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:30:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:30:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:30:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:36 smithi167 ceph-mon[29496]: pgmap v1514: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:36 smithi167 ceph-mon[37701]: pgmap v1514: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:36 smithi179 ceph-mon[33312]: pgmap v1514: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:38 smithi167 ceph-mon[29496]: pgmap v1515: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:38.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:38 smithi167 ceph-mon[37701]: pgmap v1515: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:38 smithi179 ceph-mon[33312]: pgmap v1515: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:40.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:30:40 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:30:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:40 smithi167 ceph-mon[29496]: pgmap v1516: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:40 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:30:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:40 smithi167 ceph-mon[37701]: pgmap v1516: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:40 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:30:41.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:40 smithi179 ceph-mon[33312]: pgmap v1516: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:40 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:30:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:42 smithi167 ceph-mon[29496]: pgmap v1517: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:42 smithi167 ceph-mon[37701]: pgmap v1517: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:42 smithi179 ceph-mon[33312]: pgmap v1517: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:44 smithi167 ceph-mon[29496]: pgmap v1518: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:44 smithi167 ceph-mon[37701]: pgmap v1518: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:45.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:44 smithi179 ceph-mon[33312]: pgmap v1518: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:30:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:30:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:30:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:46 smithi167 ceph-mon[29496]: pgmap v1519: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:46 smithi167 ceph-mon[37701]: pgmap v1519: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:46 smithi179 ceph-mon[33312]: pgmap v1519: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:48 smithi167 ceph-mon[37701]: pgmap v1520: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:48 smithi167 ceph-mon[29496]: pgmap v1520: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:49.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:48 smithi179 ceph-mon[33312]: pgmap v1520: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:50.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:30:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:30:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:50 smithi167 ceph-mon[37701]: pgmap v1521: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:50 smithi167 ceph-mon[29496]: pgmap v1521: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:30:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:50 smithi179 ceph-mon[33312]: pgmap v1521: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:52 smithi167 ceph-mon[29496]: pgmap v1522: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:52 smithi167 ceph-mon[37701]: pgmap v1522: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:53.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:52 smithi179 ceph-mon[33312]: pgmap v1522: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:54 smithi167 ceph-mon[37701]: pgmap v1523: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:54 smithi167 ceph-mon[29496]: pgmap v1523: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:54 smithi179 ceph-mon[33312]: pgmap v1523: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:30:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:30:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:30:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:30:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:30:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:30:56.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:30:57.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:56 smithi179 ceph-mon[33312]: pgmap v1524: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:57.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:56 smithi167 ceph-mon[29496]: pgmap v1524: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:57.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:56 smithi167 ceph-mon[37701]: pgmap v1524: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:58.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:58 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:30:58.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:58 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:30:58.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:58 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:30:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:30:59 smithi167 ceph-mon[29496]: pgmap v1525: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:30:59 smithi167 ceph-mon[37701]: pgmap v1525: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:30:59.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:30:59 smithi179 ceph-mon[33312]: pgmap v1525: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:00.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:31:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:31:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:00 smithi167 ceph-mon[29496]: pgmap v1526: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:00 smithi167 ceph-mon[37701]: pgmap v1526: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:00 smithi179 ceph-mon[33312]: pgmap v1526: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:02 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:31:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:02 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:31:03.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:02 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:31:03.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:02 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:31:03.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:02 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:31:03.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:02 smithi167 ceph-mon[29496]: pgmap v1527: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:02 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:31:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:02 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:31:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:02 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:31:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:02 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:31:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:02 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:31:03.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:02 smithi167 ceph-mon[37701]: pgmap v1527: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:02 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:31:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:02 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:31:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:02 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:31:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:02 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:31:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:02 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:31:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:02 smithi179 ceph-mon[33312]: pgmap v1527: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:04 smithi167 ceph-mon[37701]: pgmap v1528: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:04 smithi167 ceph-mon[29496]: pgmap v1528: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:05.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:04 smithi179 ceph-mon[33312]: pgmap v1528: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:31:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:31:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:31:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:06 smithi167 ceph-mon[37701]: pgmap v1529: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:06 smithi167 ceph-mon[29496]: pgmap v1529: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:06 smithi179 ceph-mon[33312]: pgmap v1529: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:08 smithi167 ceph-mon[37701]: pgmap v1530: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:08 smithi167 ceph-mon[29496]: pgmap v1530: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:09.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:08 smithi179 ceph-mon[33312]: pgmap v1530: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:10.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:31:10 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:31:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:10 smithi167 ceph-mon[37701]: pgmap v1531: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:10 smithi167 ceph-mon[29496]: pgmap v1531: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:10 smithi179 ceph-mon[33312]: pgmap v1531: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:31:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:31:12.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:31:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:12 smithi167 ceph-mon[37701]: pgmap v1532: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:12 smithi167 ceph-mon[29496]: pgmap v1532: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:13.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:12 smithi179 ceph-mon[33312]: pgmap v1532: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:14 smithi167 ceph-mon[37701]: pgmap v1533: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:14 smithi167 ceph-mon[29496]: pgmap v1533: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:15.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:14 smithi179 ceph-mon[33312]: pgmap v1533: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:31:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:31:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:31:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:16 smithi167 ceph-mon[29496]: pgmap v1534: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:16 smithi167 ceph-mon[37701]: pgmap v1534: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:17.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:16 smithi179 ceph-mon[33312]: pgmap v1534: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:18 smithi167 ceph-mon[37701]: pgmap v1535: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:18 smithi167 ceph-mon[29496]: pgmap v1535: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:19.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:18 smithi179 ceph-mon[33312]: pgmap v1535: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:20.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:31:20 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:31:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:20 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:20 smithi167 ceph-mon[37701]: pgmap v1536: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:20 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:20.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:20 smithi167 ceph-mon[29496]: pgmap v1536: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:20 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:21.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:20 smithi179 ceph-mon[33312]: pgmap v1536: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:22 smithi167 ceph-mon[37701]: pgmap v1537: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:22 smithi167 ceph-mon[29496]: pgmap v1537: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:23.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:22 smithi179 ceph-mon[33312]: pgmap v1537: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:24 smithi167 ceph-mon[37701]: pgmap v1538: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:24 smithi167 ceph-mon[29496]: pgmap v1538: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:24 smithi179 ceph-mon[33312]: pgmap v1538: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:31:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:31:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:31:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:31:26.181 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:31:26.181 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:31:27.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:26 smithi179 ceph-mon[33312]: pgmap v1539: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:27.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:26 smithi167 ceph-mon[37701]: pgmap v1539: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:27.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:26 smithi167 ceph-mon[29496]: pgmap v1539: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:28 smithi167 ceph-mon[37701]: pgmap v1540: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:28 smithi167 ceph-mon[29496]: pgmap v1540: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:29.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:28 smithi179 ceph-mon[33312]: pgmap v1540: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:30.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:31:30 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:31:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:30 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:30 smithi167 ceph-mon[37701]: pgmap v1541: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:30 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:30 smithi167 ceph-mon[29496]: pgmap v1541: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:30 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:30 smithi179 ceph-mon[33312]: pgmap v1541: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:32 smithi167 ceph-mon[37701]: pgmap v1542: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:31:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:32 smithi167 ceph-mon[29496]: pgmap v1542: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:31:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:32 smithi179 ceph-mon[33312]: pgmap v1542: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:31:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:34 smithi167 ceph-mon[29496]: pgmap v1543: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:34 smithi167 ceph-mon[37701]: pgmap v1543: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:34 smithi179 ceph-mon[33312]: pgmap v1543: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:31:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:31:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:31:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:36 smithi167 ceph-mon[37701]: pgmap v1544: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:31:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:36 smithi167 ceph-mon[29496]: pgmap v1544: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:31:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:36 smithi179 ceph-mon[33312]: pgmap v1544: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:31:38.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:38 smithi167 ceph-mon[37701]: pgmap v1545: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:31:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:38 smithi167 ceph-mon[29496]: pgmap v1545: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:31:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:38 smithi179 ceph-mon[33312]: pgmap v1545: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:31:40.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:31:40 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:31:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:40 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:40 smithi167 ceph-mon[37701]: pgmap v1546: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:40 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:40 smithi167 ceph-mon[29496]: pgmap v1546: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:41.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:40 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:40 smithi179 ceph-mon[33312]: pgmap v1546: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:41 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:31:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:41 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:31:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:41 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:31:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:42 smithi167 ceph-mon[37701]: pgmap v1547: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:42 smithi167 ceph-mon[29496]: pgmap v1547: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:42 smithi179 ceph-mon[33312]: pgmap v1547: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:44 smithi167 ceph-mon[37701]: pgmap v1548: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:44 smithi167 ceph-mon[29496]: pgmap v1548: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:45.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:44 smithi179 ceph-mon[33312]: pgmap v1548: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:31:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:31:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:31:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:46 smithi167 ceph-mon[29496]: pgmap v1549: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:46 smithi167 ceph-mon[37701]: pgmap v1549: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:47.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:46 smithi179 ceph-mon[33312]: pgmap v1549: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:48 smithi167 ceph-mon[29496]: pgmap v1550: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:48 smithi167 ceph-mon[37701]: pgmap v1550: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:49.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:48 smithi179 ceph-mon[33312]: pgmap v1550: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:50.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:31:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:31:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:50 smithi167 ceph-mon[37701]: pgmap v1551: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:50 smithi167 ceph-mon[29496]: pgmap v1551: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:31:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:50 smithi179 ceph-mon[33312]: pgmap v1551: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:52 smithi167 ceph-mon[37701]: pgmap v1552: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:52 smithi167 ceph-mon[29496]: pgmap v1552: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:53.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:52 smithi179 ceph-mon[33312]: pgmap v1552: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:54 smithi167 ceph-mon[29496]: pgmap v1553: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:54 smithi167 ceph-mon[37701]: pgmap v1553: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:55.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:54 smithi179 ceph-mon[33312]: pgmap v1553: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:31:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:31:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:31:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:31:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:55 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:31:56.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:55 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:31:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:55 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:31:57.038 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:56 smithi179 ceph-mon[33312]: pgmap v1554: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:57.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:56 smithi167 ceph-mon[29496]: pgmap v1554: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:57.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:56 smithi167 ceph-mon[37701]: pgmap v1554: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:59.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:31:59 smithi167 ceph-mon[29496]: pgmap v1555: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:59.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:31:59 smithi167 ceph-mon[37701]: pgmap v1555: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:31:59.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:31:59 smithi179 ceph-mon[33312]: pgmap v1555: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:00.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:32:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:32:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:00 smithi167 ceph-mon[29496]: pgmap v1556: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:00 smithi167 ceph-mon[37701]: pgmap v1556: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:01.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:00 smithi179 ceph-mon[33312]: pgmap v1556: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:02.386 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:02 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:32:02.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:02 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:32:02.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:02 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:32:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:03 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:32:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:03 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:32:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:03 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:32:03.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:03 smithi179 ceph-mon[33312]: pgmap v1557: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:32:03.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:03 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:32:03.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:03 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:32:03.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:03 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:32:03.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:03 smithi167 ceph-mon[37701]: pgmap v1557: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:32:03.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:03 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:32:03.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:03 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:32:03.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:03 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:32:03.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:03 smithi167 ceph-mon[29496]: pgmap v1557: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:32:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:04 smithi167 ceph-mon[37701]: pgmap v1558: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:04 smithi167 ceph-mon[29496]: pgmap v1558: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:05.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:04 smithi179 ceph-mon[33312]: pgmap v1558: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:32:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:32:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:32:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:06 smithi167 ceph-mon[37701]: pgmap v1559: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:06 smithi167 ceph-mon[29496]: pgmap v1559: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:07.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:06 smithi179 ceph-mon[33312]: pgmap v1559: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:08 smithi167 ceph-mon[37701]: pgmap v1560: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:08 smithi167 ceph-mon[29496]: pgmap v1560: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:09.136 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:08 smithi179 ceph-mon[33312]: pgmap v1560: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:10.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:32:10 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:32:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:10 smithi167 ceph-mon[37701]: pgmap v1561: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:10 smithi167 ceph-mon[29496]: pgmap v1561: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:10 smithi179 ceph-mon[33312]: pgmap v1561: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:32:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:32:12.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:32:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:12 smithi167 ceph-mon[29496]: pgmap v1562: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:12 smithi167 ceph-mon[37701]: pgmap v1562: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:13.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:12 smithi179 ceph-mon[33312]: pgmap v1562: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:14 smithi167 ceph-mon[37701]: pgmap v1563: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:14 smithi167 ceph-mon[29496]: pgmap v1563: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:15.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:14 smithi179 ceph-mon[33312]: pgmap v1563: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:32:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:32:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:32:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:17 smithi167 ceph-mon[37701]: pgmap v1564: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:17 smithi167 ceph-mon[29496]: pgmap v1564: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:18.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:17 smithi179 ceph-mon[33312]: pgmap v1564: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:20.411 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:32:20 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:32:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:20 smithi167 ceph-mon[37701]: pgmap v1565: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:20 smithi167 ceph-mon[29496]: pgmap v1565: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:20 smithi179 ceph-mon[33312]: pgmap v1565: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:21 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:21.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:21 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:21 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:22 smithi167 ceph-mon[37701]: pgmap v1566: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:22.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:22 smithi167 ceph-mon[29496]: pgmap v1566: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:22 smithi179 ceph-mon[33312]: pgmap v1566: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:24 smithi167 ceph-mon[29496]: pgmap v1567: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:24 smithi167 ceph-mon[37701]: pgmap v1567: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:24 smithi179 ceph-mon[33312]: pgmap v1567: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:32:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:32:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:32:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:26 smithi167 ceph-mon[37701]: pgmap v1568: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:26 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:32:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:26 smithi167 ceph-mon[29496]: pgmap v1568: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:26 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:32:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:26 smithi179 ceph-mon[33312]: pgmap v1568: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:26 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:32:28.689 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:28 smithi179 ceph-mon[33312]: pgmap v1569: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:28.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:28 smithi167 ceph-mon[37701]: pgmap v1569: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:28.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:28 smithi167 ceph-mon[29496]: pgmap v1569: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:30.432 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:32:30 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:32:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:30 smithi167 ceph-mon[37701]: pgmap v1570: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:30 smithi167 ceph-mon[29496]: pgmap v1570: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:30 smithi179 ceph-mon[33312]: pgmap v1570: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:31 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:31 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:31 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:32 smithi167 ceph-mon[37701]: pgmap v1571: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:32 smithi167 ceph-mon[29496]: pgmap v1571: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:32 smithi179 ceph-mon[33312]: pgmap v1571: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:34 smithi167 ceph-mon[29496]: pgmap v1572: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:34 smithi167 ceph-mon[37701]: pgmap v1572: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:34 smithi179 ceph-mon[33312]: pgmap v1572: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:32:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:32:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:32:36.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:36 smithi167 ceph-mon[29496]: pgmap v1573: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:36 smithi167 ceph-mon[37701]: pgmap v1573: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:36.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:36 smithi179 ceph-mon[33312]: pgmap v1573: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:38.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:38 smithi167 ceph-mon[29496]: pgmap v1574: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:38 smithi167 ceph-mon[37701]: pgmap v1574: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:38 smithi179 ceph-mon[33312]: pgmap v1574: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:40.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:32:40 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:32:40.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:40 smithi167 ceph-mon[37701]: pgmap v1575: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:40.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:40 smithi167 ceph-mon[29496]: pgmap v1575: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:40.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:40 smithi179 ceph-mon[33312]: pgmap v1575: 196 pgs: 196 active+clean; 583 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:41 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:41.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:41 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:32:41.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:41 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:41.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:41 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:32:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:41 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:41 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:32:42.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:42 smithi167 ceph-mon[37701]: pgmap v1576: 196 pgs: 196 active+clean; 583 KiB data, 836 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:42.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:42 smithi167 ceph-mon[29496]: pgmap v1576: 196 pgs: 196 active+clean; 583 KiB data, 836 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:42 smithi179 ceph-mon[33312]: pgmap v1576: 196 pgs: 196 active+clean; 583 KiB data, 836 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:44.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:44 smithi167 ceph-mon[37701]: pgmap v1577: 196 pgs: 196 active+clean; 583 KiB data, 836 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:44.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:44 smithi167 ceph-mon[29496]: pgmap v1577: 196 pgs: 196 active+clean; 583 KiB data, 836 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:44 smithi179 ceph-mon[33312]: pgmap v1577: 196 pgs: 196 active+clean; 583 KiB data, 836 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:32:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:32:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:32:46.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:46 smithi167 ceph-mon[37701]: pgmap v1578: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:46.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:46 smithi167 ceph-mon[29496]: pgmap v1578: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:46 smithi179 ceph-mon[33312]: pgmap v1578: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:48.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:48 smithi167 ceph-mon[37701]: pgmap v1579: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:48.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:48 smithi167 ceph-mon[29496]: pgmap v1579: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:48 smithi179 ceph-mon[33312]: pgmap v1579: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:50.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:32:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:32:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:50 smithi179 ceph-mon[33312]: pgmap v1580: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:50 smithi167 ceph-mon[37701]: pgmap v1580: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:50 smithi167 ceph-mon[29496]: pgmap v1580: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:51 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:51 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:51 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:32:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:52 smithi179 ceph-mon[33312]: pgmap v1581: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:52 smithi167 ceph-mon[29496]: pgmap v1581: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:52 smithi167 ceph-mon[37701]: pgmap v1581: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-18T13:32:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:54 smithi179 ceph-mon[33312]: pgmap v1582: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:54 smithi167 ceph-mon[29496]: pgmap v1582: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:54 smithi167 ceph-mon[37701]: pgmap v1582: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:32:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:32:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:32:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:56 smithi179 ceph-mon[33312]: pgmap v1583: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:32:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:56 smithi167 ceph-mon[37701]: pgmap v1583: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:32:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:56 smithi167 ceph-mon[29496]: pgmap v1583: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:32:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:32:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:32:58 smithi179 ceph-mon[33312]: pgmap v1584: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:32:58 smithi167 ceph-mon[37701]: pgmap v1584: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:32:58.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:32:58 smithi167 ceph-mon[29496]: pgmap v1584: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:00.499 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:33:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:33:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:00 smithi179 ceph-mon[33312]: pgmap v1585: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:00 smithi167 ceph-mon[37701]: pgmap v1585: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:00 smithi167 ceph-mon[29496]: pgmap v1585: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:01 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:01 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:01 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:02.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:02 smithi179 ceph-mon[33312]: pgmap v1586: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:02 smithi167 ceph-mon[37701]: pgmap v1586: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:02 smithi167 ceph-mon[29496]: pgmap v1586: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:03.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:03 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:33:03.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:03 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:33:03.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:03 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:33:04.729 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:04 smithi179 ceph-mon[33312]: pgmap v1587: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:04 smithi167 ceph-mon[37701]: pgmap v1587: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:04 smithi167 ceph-mon[29496]: pgmap v1587: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:33:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:33:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:33:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:06 smithi179 ceph-mon[33312]: pgmap v1588: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:06 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:33:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:06 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:33:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:06 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:33:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:06 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:33:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:06 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:33:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:06 smithi167 ceph-mon[37701]: pgmap v1588: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:06 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:33:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:06 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:33:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:06 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:33:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:06 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:33:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:06 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:33:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:06 smithi167 ceph-mon[29496]: pgmap v1588: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:06.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:06 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:33:06.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:06 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:33:06.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:06 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:33:06.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:06 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:33:06.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:06 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:33:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:08 smithi179 ceph-mon[33312]: pgmap v1589: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:08 smithi167 ceph-mon[29496]: pgmap v1589: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:08 smithi167 ceph-mon[37701]: pgmap v1589: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:10.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:33:10 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:33:10.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:10 smithi179 ceph-mon[33312]: pgmap v1590: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:10.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:10 smithi167 ceph-mon[29496]: pgmap v1590: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:10 smithi167 ceph-mon[37701]: pgmap v1590: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:11 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:33:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:11 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:33:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:11 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:11.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:33:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:12 smithi179 ceph-mon[33312]: pgmap v1591: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:12 smithi167 ceph-mon[29496]: pgmap v1591: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:12 smithi167 ceph-mon[37701]: pgmap v1591: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:14 smithi179 ceph-mon[33312]: pgmap v1592: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:14 smithi167 ceph-mon[37701]: pgmap v1592: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:14 smithi167 ceph-mon[29496]: pgmap v1592: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:33:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:33:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:33:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:16 smithi179 ceph-mon[33312]: pgmap v1593: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:16.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:16 smithi167 ceph-mon[37701]: pgmap v1593: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:16 smithi167 ceph-mon[29496]: pgmap v1593: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:18 smithi179 ceph-mon[33312]: pgmap v1594: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:18 smithi167 ceph-mon[37701]: pgmap v1594: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:18.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:18 smithi167 ceph-mon[29496]: pgmap v1594: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:20.543 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:33:20 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:33:20.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:20 smithi179 ceph-mon[33312]: pgmap v1595: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:20.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:20 smithi167 ceph-mon[37701]: pgmap v1595: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:20.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:20 smithi167 ceph-mon[29496]: pgmap v1595: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:21 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:21 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:21 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:22 smithi179 ceph-mon[33312]: pgmap v1596: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:22 smithi167 ceph-mon[37701]: pgmap v1596: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:22 smithi167 ceph-mon[29496]: pgmap v1596: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:24 smithi179 ceph-mon[33312]: pgmap v1597: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:24 smithi167 ceph-mon[37701]: pgmap v1597: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:24.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:24 smithi167 ceph-mon[29496]: pgmap v1597: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:33:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:33:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:33:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:26 smithi179 ceph-mon[33312]: pgmap v1598: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:26 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:33:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:26 smithi167 ceph-mon[37701]: pgmap v1598: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:26 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:33:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:26 smithi167 ceph-mon[29496]: pgmap v1598: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:26 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:33:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:28 smithi179 ceph-mon[33312]: pgmap v1599: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:33:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:28 smithi167 ceph-mon[37701]: pgmap v1599: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:33:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:28 smithi167 ceph-mon[29496]: pgmap v1599: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:33:30.562 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:33:30 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:33:30.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:30 smithi179 ceph-mon[33312]: pgmap v1600: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:30 smithi167 ceph-mon[37701]: pgmap v1600: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:30 smithi167 ceph-mon[29496]: pgmap v1600: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:31 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:31 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:31 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:32 smithi179 ceph-mon[33312]: pgmap v1601: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:33:32.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:32 smithi167 ceph-mon[37701]: pgmap v1601: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:33:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:32 smithi167 ceph-mon[29496]: pgmap v1601: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:33:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:34 smithi179 ceph-mon[33312]: pgmap v1602: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:34 smithi167 ceph-mon[37701]: pgmap v1602: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:34 smithi167 ceph-mon[29496]: pgmap v1602: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:33:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:33:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:33:36.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:36 smithi179 ceph-mon[33312]: pgmap v1603: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:36 smithi167 ceph-mon[37701]: pgmap v1603: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:36 smithi167 ceph-mon[29496]: pgmap v1603: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:38 smithi179 ceph-mon[33312]: pgmap v1604: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:33:38.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:38 smithi167 ceph-mon[37701]: pgmap v1604: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:33:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:38 smithi167 ceph-mon[29496]: pgmap v1604: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:33:40.581 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:33:40 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:33:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:40 smithi179 ceph-mon[33312]: pgmap v1605: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:40 smithi167 ceph-mon[37701]: pgmap v1605: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:40 smithi167 ceph-mon[29496]: pgmap v1605: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:41 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:41 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:33:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:41 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:41 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:33:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:41 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:41 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:33:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:42 smithi179 ceph-mon[33312]: pgmap v1606: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:42 smithi167 ceph-mon[37701]: pgmap v1606: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:42 smithi167 ceph-mon[29496]: pgmap v1606: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:44 smithi179 ceph-mon[33312]: pgmap v1607: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:44 smithi167 ceph-mon[37701]: pgmap v1607: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:44 smithi167 ceph-mon[29496]: pgmap v1607: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:33:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:33:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:33:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:46 smithi179 ceph-mon[33312]: pgmap v1608: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:46 smithi167 ceph-mon[29496]: pgmap v1608: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:46 smithi167 ceph-mon[37701]: pgmap v1608: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:48 smithi179 ceph-mon[33312]: pgmap v1609: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:48 smithi167 ceph-mon[29496]: pgmap v1609: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:48 smithi167 ceph-mon[37701]: pgmap v1609: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:50.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:33:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:33:50.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:50 smithi179 ceph-mon[33312]: pgmap v1610: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:50 smithi167 ceph-mon[29496]: pgmap v1610: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:50 smithi167 ceph-mon[37701]: pgmap v1610: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:51 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:51 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:51 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:33:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:52 smithi179 ceph-mon[33312]: pgmap v1611: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:52 smithi167 ceph-mon[37701]: pgmap v1611: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:52 smithi167 ceph-mon[29496]: pgmap v1611: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:54 smithi179 ceph-mon[33312]: pgmap v1612: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:54 smithi167 ceph-mon[37701]: pgmap v1612: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:54 smithi167 ceph-mon[29496]: pgmap v1612: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:33:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:33:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:33:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:56 smithi179 ceph-mon[33312]: pgmap v1613: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:33:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:56 smithi167 ceph-mon[37701]: pgmap v1613: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:33:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:56 smithi167 ceph-mon[29496]: pgmap v1613: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:33:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:33:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:33:58 smithi179 ceph-mon[33312]: pgmap v1614: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:33:58 smithi167 ceph-mon[37701]: pgmap v1614: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:33:58.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:33:58 smithi167 ceph-mon[29496]: pgmap v1614: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:00.577 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:34:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:34:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:00 smithi179 ceph-mon[33312]: pgmap v1615: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:00.887 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:34:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=cleanup t=2024-06-18T13:34:00.578199982Z level=info msg="Completed cleanup jobs" duration=1.052868ms 2024-06-18T13:34:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:00 smithi167 ceph-mon[29496]: pgmap v1615: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:00 smithi167 ceph-mon[37701]: pgmap v1615: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:01 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:01 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:01 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:02 smithi179 ceph-mon[33312]: pgmap v1616: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:02 smithi167 ceph-mon[29496]: pgmap v1616: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:02 smithi167 ceph-mon[37701]: pgmap v1616: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:04 smithi179 ceph-mon[33312]: pgmap v1617: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:04 smithi167 ceph-mon[37701]: pgmap v1617: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:04.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:04 smithi167 ceph-mon[29496]: pgmap v1617: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:34:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:34:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:34:06.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:06 smithi179 ceph-mon[33312]: pgmap v1618: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:06 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:34:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:06 smithi167 ceph-mon[37701]: pgmap v1618: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:06 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:34:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:06 smithi167 ceph-mon[29496]: pgmap v1618: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:06 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:34:08.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[37701]: pgmap v1619: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:08.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:08.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:08.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:08.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:08.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:34:08.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:34:08.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:08.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[29496]: pgmap v1619: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:08.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:08.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:08.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:08.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:08.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:34:08.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:34:08.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:07 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:07 smithi179 ceph-mon[33312]: pgmap v1619: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:07 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:07 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:07 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:07 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:07 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:34:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:07 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:34:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:07 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:34:10.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:09 smithi167 ceph-mon[37701]: pgmap v1620: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:10.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:09 smithi167 ceph-mon[29496]: pgmap v1620: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:10.255 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:09 smithi179 ceph-mon[33312]: pgmap v1620: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:10.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:34:10 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:34:11.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:11.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:34:11.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:11.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:34:11.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:11.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:34:12.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:11 smithi167 ceph-mon[37701]: pgmap v1621: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:12.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:11 smithi167 ceph-mon[29496]: pgmap v1621: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:12.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:11 smithi179 ceph-mon[33312]: pgmap v1621: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:14.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:13 smithi167 ceph-mon[37701]: pgmap v1622: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:14.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:13 smithi167 ceph-mon[29496]: pgmap v1622: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:14.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:13 smithi179 ceph-mon[33312]: pgmap v1622: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:34:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:34:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:34:16.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:15 smithi167 ceph-mon[29496]: pgmap v1623: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:16.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:15 smithi167 ceph-mon[37701]: pgmap v1623: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:16.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:15 smithi179 ceph-mon[33312]: pgmap v1623: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:18.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:17 smithi167 ceph-mon[37701]: pgmap v1624: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:34:18.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:17 smithi167 ceph-mon[29496]: pgmap v1624: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:34:18.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:17 smithi179 ceph-mon[33312]: pgmap v1624: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:34:20.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:19 smithi167 ceph-mon[29496]: pgmap v1625: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:20.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:19 smithi167 ceph-mon[37701]: pgmap v1625: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:20.266 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:19 smithi179 ceph-mon[33312]: pgmap v1625: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:20.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:34:20 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:34:21.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:20 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:21.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:20 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:21.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:20 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:22.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:21 smithi179 ceph-mon[33312]: pgmap v1626: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:22.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:21 smithi167 ceph-mon[37701]: pgmap v1626: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:22.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:21 smithi167 ceph-mon[29496]: pgmap v1626: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:24.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:23 smithi179 ceph-mon[33312]: pgmap v1627: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:24.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:23 smithi167 ceph-mon[37701]: pgmap v1627: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:24.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:23 smithi167 ceph-mon[29496]: pgmap v1627: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:34:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:34:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:34:26.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:25 smithi179 ceph-mon[33312]: pgmap v1628: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:26.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:25 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:34:26.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:25 smithi167 ceph-mon[37701]: pgmap v1628: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:26.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:25 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:34:26.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:25 smithi167 ceph-mon[29496]: pgmap v1628: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:26.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:25 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:34:28.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:27 smithi179 ceph-mon[33312]: pgmap v1629: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:28.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:27 smithi167 ceph-mon[29496]: pgmap v1629: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:28.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:27 smithi167 ceph-mon[37701]: pgmap v1629: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:30.277 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:29 smithi179 ceph-mon[33312]: pgmap v1630: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:30.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:29 smithi167 ceph-mon[37701]: pgmap v1630: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:30.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:29 smithi167 ceph-mon[29496]: pgmap v1630: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:30.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:34:30 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:34:31.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:31 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:31.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:31 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:31.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:31 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:32.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:32 smithi179 ceph-mon[33312]: pgmap v1631: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:32.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:32 smithi167 ceph-mon[37701]: pgmap v1631: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:32.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:32 smithi167 ceph-mon[29496]: pgmap v1631: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:34.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:34 smithi179 ceph-mon[33312]: pgmap v1632: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:34.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:34 smithi167 ceph-mon[37701]: pgmap v1632: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:34.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:34 smithi167 ceph-mon[29496]: pgmap v1632: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:34:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:34:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:34:36.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:36 smithi179 ceph-mon[33312]: pgmap v1633: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:36.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:36 smithi167 ceph-mon[37701]: pgmap v1633: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:36.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:36 smithi167 ceph-mon[29496]: pgmap v1633: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:38.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:38 smithi179 ceph-mon[33312]: pgmap v1634: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:34:38.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:38 smithi167 ceph-mon[37701]: pgmap v1634: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:34:38.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:38 smithi167 ceph-mon[29496]: pgmap v1634: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:34:40.288 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:40 smithi179 ceph-mon[33312]: pgmap v1635: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:40.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:40 smithi167 ceph-mon[37701]: pgmap v1635: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:40.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:40 smithi167 ceph-mon[29496]: pgmap v1635: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:40.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:34:40 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:34:41.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:41 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:41.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:41 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:34:41.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:41 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:41.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:41 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:34:41.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:41 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:41.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:41 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:34:42.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:42 smithi179 ceph-mon[33312]: pgmap v1636: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:34:42.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:42 smithi167 ceph-mon[37701]: pgmap v1636: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:34:42.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:42 smithi167 ceph-mon[29496]: pgmap v1636: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:34:44.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:44 smithi179 ceph-mon[33312]: pgmap v1637: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:44.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:44 smithi167 ceph-mon[37701]: pgmap v1637: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:44.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:44 smithi167 ceph-mon[29496]: pgmap v1637: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:45.378 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:34:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:34:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:34:46.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:46 smithi179 ceph-mon[33312]: pgmap v1638: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:46.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:46 smithi167 ceph-mon[37701]: pgmap v1638: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:46.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:46 smithi167 ceph-mon[29496]: pgmap v1638: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:48.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:48 smithi179 ceph-mon[33312]: pgmap v1639: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:48.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:48 smithi167 ceph-mon[37701]: pgmap v1639: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:48.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:48 smithi167 ceph-mon[29496]: pgmap v1639: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:50.299 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:50 smithi179 ceph-mon[33312]: pgmap v1640: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:50.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:50 smithi167 ceph-mon[37701]: pgmap v1640: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:50.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:50 smithi167 ceph-mon[29496]: pgmap v1640: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:50.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:34:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:34:51.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:51 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:51.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:51 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:51.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:51 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:34:52.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:52 smithi179 ceph-mon[33312]: pgmap v1641: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:34:52.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:52 smithi167 ceph-mon[37701]: pgmap v1641: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:34:52.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:52 smithi167 ceph-mon[29496]: pgmap v1641: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:34:54.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:54 smithi179 ceph-mon[33312]: pgmap v1642: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:54.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:54 smithi167 ceph-mon[29496]: pgmap v1642: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:54.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:54 smithi167 ceph-mon[37701]: pgmap v1642: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:34:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:34:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:34:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:34:56.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:56 smithi179 ceph-mon[33312]: pgmap v1643: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:56.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:34:56.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:56 smithi167 ceph-mon[29496]: pgmap v1643: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:56.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:34:56.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:56 smithi167 ceph-mon[37701]: pgmap v1643: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:34:56.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:34:58.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:34:58 smithi179 ceph-mon[33312]: pgmap v1644: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:34:58.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:34:58 smithi167 ceph-mon[37701]: pgmap v1644: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:34:58.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:34:58 smithi167 ceph-mon[29496]: pgmap v1644: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:35:00.310 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:00 smithi179 ceph-mon[33312]: pgmap v1645: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:00.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:00 smithi167 ceph-mon[37701]: pgmap v1645: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:00.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:00 smithi167 ceph-mon[29496]: pgmap v1645: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:00.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:35:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:35:01.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:01 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:01.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:01 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:01.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:01 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:02.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:02 smithi179 ceph-mon[33312]: pgmap v1646: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:35:02.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:02 smithi167 ceph-mon[37701]: pgmap v1646: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:35:02.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:02 smithi167 ceph-mon[29496]: pgmap v1646: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:35:04.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:04 smithi179 ceph-mon[33312]: pgmap v1647: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:35:04.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:04 smithi179 ceph-mon[33312]: osdmap e758: 8 total, 8 up, 8 in 2024-06-18T13:35:04.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:04 smithi167 ceph-mon[37701]: pgmap v1647: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:35:04.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:04 smithi167 ceph-mon[37701]: osdmap e758: 8 total, 8 up, 8 in 2024-06-18T13:35:04.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:04 smithi167 ceph-mon[29496]: pgmap v1647: 196 pgs: 196 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:35:04.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:04 smithi167 ceph-mon[29496]: osdmap e758: 8 total, 8 up, 8 in 2024-06-18T13:35:05.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:05 smithi179 ceph-mon[33312]: osdmap e759: 8 total, 8 up, 8 in 2024-06-18T13:35:05.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:05 smithi167 ceph-mon[37701]: osdmap e759: 8 total, 8 up, 8 in 2024-06-18T13:35:05.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:05 smithi167 ceph-mon[29496]: osdmap e759: 8 total, 8 up, 8 in 2024-06-18T13:35:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:35:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:35:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:35:06.102 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_1_[74999]: starting. 2024-06-18T13:35:06.102 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_1_[74999]: creating pool ceph_test_rados_delete_pools_parallel.smithi167-74857 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_1_[74999]: created object 0... 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_1_[74999]: created object 25... 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_1_[74999]: created object 49... 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_1_[74999]: finishing. 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_1_[74999]: shutting down. 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_2_[75000]: starting. 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_2_[75000]: deleting pool ceph_test_rados_delete_pools_parallel.smithi167-74857 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_2_[75000]: shutting down. 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: ******************************* 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_3_[149577]: starting. 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_3_[149577]: creating pool ceph_test_rados_delete_pools_parallel.smithi167-74857 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_3_[149577]: created object 0... 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_3_[149577]: created object 25... 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_3_[149577]: created object 49... 2024-06-18T13:35:06.103 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_3_[149577]: finishing. 2024-06-18T13:35:06.104 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_3_[149577]: shutting down. 2024-06-18T13:35:06.104 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: ******************************* 2024-06-18T13:35:06.104 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_5_[149579]: starting. 2024-06-18T13:35:06.104 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_5_[149579]: listing objects. 2024-06-18T13:35:06.104 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_5_[149579]: listed object 0... 2024-06-18T13:35:06.104 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_5_[149579]: listed object 25... 2024-06-18T13:35:06.104 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_5_[149579]: saw 50 objects 2024-06-18T13:35:06.104 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_5_[149579]: shutting down. 2024-06-18T13:35:06.104 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: ******************************* 2024-06-18T13:35:06.104 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_4_[149578]: starting. 2024-06-18T13:35:06.104 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_4_[149578]: deleting pool ceph_test_rados_delete_pools_parallel.smithi167-74857 2024-06-18T13:35:06.104 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: process_4_[149578]: shutting down. 2024-06-18T13:35:06.104 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: ******************************* 2024-06-18T13:35:06.105 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: ******************************* 2024-06-18T13:35:06.105 INFO:tasks.workunit.client.0.smithi167.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-06-18T13:35:06.109 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.109 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74419 2024-06-18T13:35:06.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74419 2024-06-18T13:35:06.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74303 2024-06-18T13:35:06.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74303 2024-06-18T13:35:06.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74036 2024-06-18T13:35:06.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74036 2024-06-18T13:35:06.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74353 2024-06-18T13:35:06.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74353 2024-06-18T13:35:06.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74881 2024-06-18T13:35:06.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74881 2024-06-18T13:35:06.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74954 2024-06-18T13:35:06.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74954 2024-06-18T13:35:06.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74058 2024-06-18T13:35:06.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74058 2024-06-18T13:35:06.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74712 2024-06-18T13:35:06.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74712 2024-06-18T13:35:06.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74614 2024-06-18T13:35:06.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74614 2024-06-18T13:35:06.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=73921 2024-06-18T13:35:06.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 73921 2024-06-18T13:35:06.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=73985 2024-06-18T13:35:06.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 73985 2024-06-18T13:35:06.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74625 2024-06-18T13:35:06.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74625 2024-06-18T13:35:06.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=73912 2024-06-18T13:35:06.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 73912 2024-06-18T13:35:06.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=73932 2024-06-18T13:35:06.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 73932 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75086 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 75086 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74235 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74235 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75217 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 75217 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74317 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74317 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74826 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74826 2024-06-18T13:35:06.113 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75239 2024-06-18T13:35:06.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 75239 2024-06-18T13:35:06.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74264 2024-06-18T13:35:06.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74264 2024-06-18T13:35:06.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74656 2024-06-18T13:35:06.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74656 2024-06-18T13:35:06.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75034 2024-06-18T13:35:06.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 75034 2024-06-18T13:35:06.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74765 2024-06-18T13:35:06.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74765 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75192 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 75192 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74581 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74581 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74156 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74156 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75227 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 75227 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74465 2024-06-18T13:35:06.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74465 2024-06-18T13:35:06.116 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.116 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=73905 2024-06-18T13:35:06.116 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 73905 2024-06-18T13:35:06.116 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.116 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=73965 2024-06-18T13:35:06.116 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 73965 2024-06-18T13:35:06.116 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.116 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74382 2024-06-18T13:35:06.116 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74382 2024-06-18T13:35:06.116 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.116 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74008 2024-06-18T13:35:06.116 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74008 2024-06-18T13:35:06.116 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74502 2024-06-18T13:35:06.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74502 2024-06-18T13:35:06.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=74547 2024-06-18T13:35:06.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 74547 2024-06-18T13:35:06.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75004 2024-06-18T13:35:06.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 75004 2024-06-18T13:35:06.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ for t in "${!pids[@]}" 2024-06-18T13:35:06.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=75253 2024-06-18T13:35:06.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 75253 2024-06-18T13:35:06.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ exit 0 2024-06-18T13:35:06.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ cleanup 2024-06-18T13:35:06.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ pkill -P 73899 2024-06-18T13:35:06.302 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2024-06-18T13:35:06.303 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-18T13:35:06.303 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-18T13:35:06.335 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-06-18T13:35:06.336 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-06-18T13:35:06.337 DEBUG:teuthology.orchestra.run.smithi167:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f18f3f1cbd11e4a41c557e616d172f24deb91473 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-06-18T13:35:06.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:06 smithi179 ceph-mon[33312]: pgmap v1650: 196 pgs: 32 unknown, 164 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:06.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:06 smithi179 ceph-mon[33312]: osdmap e760: 8 total, 8 up, 8 in 2024-06-18T13:35:06.398 INFO:tasks.workunit.client.0.smithi167.stderr:++ uuidgen 2024-06-18T13:35:06.407 INFO:tasks.workunit.client.0.smithi167.stderr:+ p=2d48a5d0-5bf6-4059-9227-066b6e28e48a 2024-06-18T13:35:06.407 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool create 2d48a5d0-5bf6-4059-9227-066b6e28e48a 12 2024-06-18T13:35:06.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:06 smithi167 ceph-mon[37701]: pgmap v1650: 196 pgs: 32 unknown, 164 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:06.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:06 smithi167 ceph-mon[37701]: osdmap e760: 8 total, 8 up, 8 in 2024-06-18T13:35:06.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:06 smithi167 ceph-mon[29496]: pgmap v1650: 196 pgs: 32 unknown, 164 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:06.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:06 smithi167 ceph-mon[29496]: osdmap e760: 8 total, 8 up, 8 in 2024-06-18T13:35:06.493 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.493+0000 7f52ea97a640 1 -- 172.21.15.167:0/601445920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52e4106630 msgr2=0x7f52e410e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:06.493 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.493+0000 7f52ea97a640 1 --2- 172.21.15.167:0/601445920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52e4106630 0x7f52e410e180 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009be0 tx=0x7f52dc02bef0 comp rx=0 tx=0).stop 2024-06-18T13:35:06.494 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.493+0000 7f52ea97a640 1 -- 172.21.15.167:0/601445920 shutdown_connections 2024-06-18T13:35:06.494 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.493+0000 7f52ea97a640 1 --2- 172.21.15.167:0/601445920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52e4106630 0x7f52e410e180 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:06.494 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.493+0000 7f52ea97a640 1 --2- 172.21.15.167:0/601445920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52e4101e90 0x7f52e4105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:06.494 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.493+0000 7f52ea97a640 1 --2- 172.21.15.167:0/601445920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e41014e0 0x7f52e41018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:06.494 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.493+0000 7f52ea97a640 1 -- 172.21.15.167:0/601445920 >> 172.21.15.167:0/601445920 conn(0x7f52e40fcf90 msgr2=0x7f52e40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:35:06.494 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.493+0000 7f52ea97a640 1 -- 172.21.15.167:0/601445920 shutdown_connections 2024-06-18T13:35:06.494 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.493+0000 7f52ea97a640 1 -- 172.21.15.167:0/601445920 wait complete. 2024-06-18T13:35:06.494 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.493+0000 7f52ea97a640 1 Processor -- start 2024-06-18T13:35:06.494 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52ea97a640 1 -- start start 2024-06-18T13:35:06.496 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52ea97a640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52e41014e0 0x7f52e4073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:06.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52ea97a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52e4106630 0x7f52e40741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:06.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52ea97a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e4074720 0x7f52e41a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:06.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52ea97a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f52e4110aa0 con 0x7f52e4074720 2024-06-18T13:35:06.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52ea97a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f52e4110920 con 0x7f52e4106630 2024-06-18T13:35:06.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52ea97a640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f52e4110c20 con 0x7f52e41014e0 2024-06-18T13:35:06.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52e3fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52e41014e0 0x7f52e4073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:06.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52e8f17640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e4074720 0x7f52e41a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:06.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52e3fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52e41014e0 0x7f52e4073ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:57962/0 (socket says 172.21.15.167:57962) 2024-06-18T13:35:06.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52e3fff640 1 -- 172.21.15.167:0/3127413520 learned_addr learned my addr 172.21.15.167:0/3127413520 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:35:06.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52e37fe640 1 --2- 172.21.15.167:0/3127413520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52e4106630 0x7f52e40741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:06.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52e3fff640 1 -- 172.21.15.167:0/3127413520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52e4106630 msgr2=0x7f52e40741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:06.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52e3fff640 1 --2- 172.21.15.167:0/3127413520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52e4106630 0x7f52e40741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:06.498 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52e3fff640 1 -- 172.21.15.167:0/3127413520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e4074720 msgr2=0x7f52e41a52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:06.498 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52e3fff640 1 --2- 172.21.15.167:0/3127413520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e4074720 0x7f52e41a52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:06.498 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52e3fff640 1 -- 172.21.15.167:0/3127413520 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e41a59d0 con 0x7f52e41014e0 2024-06-18T13:35:06.498 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52e37fe640 1 --2- 172.21.15.167:0/3127413520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52e4106630 0x7f52e40741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-18T13:35:06.498 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.494+0000 7f52e3fff640 1 --2- 172.21.15.167:0/3127413520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52e41014e0 0x7f52e4073ca0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f52d400e9c0 tx=0x7f52d400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:35:06.498 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.495+0000 7f52e17fa640 1 -- 172.21.15.167:0/3127413520 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d400cde0 con 0x7f52e41014e0 2024-06-18T13:35:06.498 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.495+0000 7f52e17fa640 1 -- 172.21.15.167:0/3127413520 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f52d40035f0 con 0x7f52e41014e0 2024-06-18T13:35:06.498 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.495+0000 7f52ea97a640 1 -- 172.21.15.167:0/3127413520 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e41a5cc0 con 0x7f52e41014e0 2024-06-18T13:35:06.498 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.495+0000 7f52e17fa640 1 -- 172.21.15.167:0/3127413520 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d40044a0 con 0x7f52e41014e0 2024-06-18T13:35:06.498 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.495+0000 7f52ea97a640 1 -- 172.21.15.167:0/3127413520 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e41a6200 con 0x7f52e41014e0 2024-06-18T13:35:06.498 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.496+0000 7f52ea97a640 1 -- 172.21.15.167:0/3127413520 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e41018c0 con 0x7f52e41014e0 2024-06-18T13:35:06.498 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.496+0000 7f52e17fa640 1 -- 172.21.15.167:0/3127413520 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f52d4004000 con 0x7f52e41014e0 2024-06-18T13:35:06.498 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.497+0000 7f52e17fa640 1 --2- 172.21.15.167:0/3127413520 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f52cc06e080 0x7f52cc070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:06.499 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.497+0000 7f52e17fa640 1 -- 172.21.15.167:0/3127413520 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(761..761 src has 257..761) ==== 8704+0+0 (secure 0 0 0) 0x7f52d408ee60 con 0x7f52e41014e0 2024-06-18T13:35:06.499 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.497+0000 7f52e37fe640 1 --2- 172.21.15.167:0/3127413520 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f52cc06e080 0x7f52cc070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:06.499 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.497+0000 7f52e37fe640 1 --2- 172.21.15.167:0/3127413520 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f52cc06e080 0x7f52cc070540 secure :-1 s=READY pgs=3487 cs=0 l=1 rev1=1 crypto rx=0x7f52e4075730 tx=0x7f52d8005cb0 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:35:06.501 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.501+0000 7f52e17fa640 1 -- 172.21.15.167:0/3127413520 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f52d405bd90 con 0x7f52e41014e0 2024-06-18T13:35:06.682 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:06.682+0000 7f52ea97a640 1 -- 172.21.15.167:0/3127413520 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12} v 0) -- 0x7f52e4101e90 con 0x7f52e41014e0 2024-06-18T13:35:07.113 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.112+0000 7f52e17fa640 1 -- 172.21.15.167:0/3127413520 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]=0 pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' created v762) ==== 176+0+0 (secure 0 0 0) 0x7f52d405fa40 con 0x7f52e41014e0 2024-06-18T13:35:07.224 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.224+0000 7f52ea97a640 1 -- 172.21.15.167:0/3127413520 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12} v 0) -- 0x7f52e4102060 con 0x7f52e41014e0 2024-06-18T13:35:07.225 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.225+0000 7f52e17fa640 1 -- 172.21.15.167:0/3127413520 <== mon.2 v2:172.21.15.167:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]=0 pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' already exists v762) ==== 183+0+0 (secure 0 0 0) 0x7f52d4064270 con 0x7f52e41014e0 2024-06-18T13:35:07.225 INFO:tasks.workunit.client.0.smithi167.stderr:pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' already exists 2024-06-18T13:35:07.228 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.227+0000 7f52ea97a640 1 -- 172.21.15.167:0/3127413520 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f52cc06e080 msgr2=0x7f52cc070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:07.228 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.227+0000 7f52ea97a640 1 --2- 172.21.15.167:0/3127413520 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f52cc06e080 0x7f52cc070540 secure :-1 s=READY pgs=3487 cs=0 l=1 rev1=1 crypto rx=0x7f52e4075730 tx=0x7f52d8005cb0 comp rx=0 tx=0).stop 2024-06-18T13:35:07.228 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.227+0000 7f52ea97a640 1 -- 172.21.15.167:0/3127413520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52e41014e0 msgr2=0x7f52e4073ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:07.228 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.227+0000 7f52ea97a640 1 --2- 172.21.15.167:0/3127413520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52e41014e0 0x7f52e4073ca0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f52d400e9c0 tx=0x7f52d400ee80 comp rx=0 tx=0).stop 2024-06-18T13:35:07.228 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.228+0000 7f52ea97a640 1 -- 172.21.15.167:0/3127413520 shutdown_connections 2024-06-18T13:35:07.228 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.228+0000 7f52ea97a640 1 --2- 172.21.15.167:0/3127413520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e4074720 0x7f52e41a52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:07.228 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.228+0000 7f52ea97a640 1 --2- 172.21.15.167:0/3127413520 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f52cc06e080 0x7f52cc070540 unknown :-1 s=CLOSED pgs=3487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:07.229 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.228+0000 7f52ea97a640 1 --2- 172.21.15.167:0/3127413520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52e4106630 0x7f52e40741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:07.229 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.228+0000 7f52ea97a640 1 --2- 172.21.15.167:0/3127413520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52e41014e0 0x7f52e4073ca0 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:07.229 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.228+0000 7f52ea97a640 1 -- 172.21.15.167:0/3127413520 >> 172.21.15.167:0/3127413520 conn(0x7f52e40fcf90 msgr2=0x7f52e40fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:35:07.229 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.228+0000 7f52ea97a640 1 -- 172.21.15.167:0/3127413520 shutdown_connections 2024-06-18T13:35:07.229 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.228+0000 7f52ea97a640 1 -- 172.21.15.167:0/3127413520 wait complete. 2024-06-18T13:35:07.237 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool set-quota 2d48a5d0-5bf6-4059-9227-066b6e28e48a max_objects 10 2024-06-18T13:35:07.329 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.326+0000 7f93f929d640 1 -- 172.21.15.167:0/1389114684 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f41051b0 msgr2=0x7f93f4105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:07.329 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.326+0000 7f93f929d640 1 --2- 172.21.15.167:0/1389114684 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f41051b0 0x7f93f4105590 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009930 tx=0x7f93e402bc30 comp rx=0 tx=0).stop 2024-06-18T13:35:07.329 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.327+0000 7f93f929d640 1 -- 172.21.15.167:0/1389114684 shutdown_connections 2024-06-18T13:35:07.330 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.327+0000 7f93f929d640 1 --2- 172.21.15.167:0/1389114684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93f4100f20 0x7f93f410d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:07.330 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.327+0000 7f93f929d640 1 --2- 172.21.15.167:0/1389114684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f4100580 0x7f93f41009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:07.330 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.327+0000 7f93f929d640 1 --2- 172.21.15.167:0/1389114684 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f41051b0 0x7f93f4105590 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:07.330 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.327+0000 7f93f929d640 1 -- 172.21.15.167:0/1389114684 >> 172.21.15.167:0/1389114684 conn(0x7f93f4075f30 msgr2=0x7f93f4076340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:35:07.330 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.327+0000 7f93f929d640 1 -- 172.21.15.167:0/1389114684 shutdown_connections 2024-06-18T13:35:07.330 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.327+0000 7f93f929d640 1 -- 172.21.15.167:0/1389114684 wait complete. 2024-06-18T13:35:07.330 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.327+0000 7f93f929d640 1 Processor -- start 2024-06-18T13:35:07.330 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.327+0000 7f93f929d640 1 -- start start 2024-06-18T13:35:07.330 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f929d640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f4100580 0x7f93f4199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:07.330 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f929d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93f4100f20 0x7f93f4199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:07.330 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f929d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f41051b0 0x7f93f419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:07.331 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f929d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f93f410f970 con 0x7f93f41051b0 2024-06-18T13:35:07.331 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f929d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f93f410f7f0 con 0x7f93f4100f20 2024-06-18T13:35:07.331 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f929d640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f93f410faf0 con 0x7f93f4100580 2024-06-18T13:35:07.331 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f2ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f4100580 0x7f93f4199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:07.331 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f41051b0 0x7f93f419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:07.331 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f2ffd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f4100580 0x7f93f4199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:57988/0 (socket says 172.21.15.167:57988) 2024-06-18T13:35:07.331 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f41051b0 0x7f93f419dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55056/0 (socket says 172.21.15.167:55056) 2024-06-18T13:35:07.331 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f2ffd640 1 -- 172.21.15.167:0/1270548309 learned_addr learned my addr 172.21.15.167:0/1270548309 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:35:07.331 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f27fc640 1 --2- 172.21.15.167:0/1270548309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93f4100f20 0x7f93f4199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:07.331 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f2ffd640 1 -- 172.21.15.167:0/1270548309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93f4100f20 msgr2=0x7f93f4199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:07.331 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f2ffd640 1 --2- 172.21.15.167:0/1270548309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93f4100f20 0x7f93f4199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:07.332 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f2ffd640 1 -- 172.21.15.167:0/1270548309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f41051b0 msgr2=0x7f93f419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:07.332 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f2ffd640 1 --2- 172.21.15.167:0/1270548309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f41051b0 0x7f93f419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:07.332 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f2ffd640 1 -- 172.21.15.167:0/1270548309 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f419e2f0 con 0x7f93f4100580 2024-06-18T13:35:07.332 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f37fe640 1 --2- 172.21.15.167:0/1270548309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f41051b0 0x7f93f419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:35:07.332 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f2ffd640 1 --2- 172.21.15.167:0/1270548309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f4100580 0x7f93f4199300 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f93e4000c00 tx=0x7f93e4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:35:07.332 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93ebfff640 1 -- 172.21.15.167:0/1270548309 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4043020 con 0x7f93f4100580 2024-06-18T13:35:07.332 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f929d640 1 -- 172.21.15.167:0/1270548309 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f419e580 con 0x7f93f4100580 2024-06-18T13:35:07.332 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93ebfff640 1 -- 172.21.15.167:0/1270548309 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f93e4004a30 con 0x7f93f4100580 2024-06-18T13:35:07.332 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93ebfff640 1 -- 172.21.15.167:0/1270548309 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e40037e0 con 0x7f93f4100580 2024-06-18T13:35:07.332 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.328+0000 7f93f929d640 1 -- 172.21.15.167:0/1270548309 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f41a5e70 con 0x7f93f4100580 2024-06-18T13:35:07.332 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.329+0000 7f93f929d640 1 -- 172.21.15.167:0/1270548309 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f41018b0 con 0x7f93f4100580 2024-06-18T13:35:07.333 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.330+0000 7f93ebfff640 1 -- 172.21.15.167:0/1270548309 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f93e4046020 con 0x7f93f4100580 2024-06-18T13:35:07.333 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.331+0000 7f93ebfff640 1 --2- 172.21.15.167:0/1270548309 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f93e006e050 0x7f93e0070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:07.333 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.331+0000 7f93ebfff640 1 -- 172.21.15.167:0/1270548309 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(762..762 src has 257..762) ==== 9079+0+0 (secure 0 0 0) 0x7f93e40b0b40 con 0x7f93f4100580 2024-06-18T13:35:07.333 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.331+0000 7f93f27fc640 1 --2- 172.21.15.167:0/1270548309 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f93e006e050 0x7f93e0070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:07.334 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.331+0000 7f93f27fc640 1 --2- 172.21.15.167:0/1270548309 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f93e006e050 0x7f93e0070510 secure :-1 s=READY pgs=3488 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0095f0 tx=0x7f93dc005f90 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:35:07.335 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.335+0000 7f93ebfff640 1 -- 172.21.15.167:0/1270548309 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f93e40b4030 con 0x7f93f4100580 2024-06-18T13:35:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:07 smithi179 ceph-mon[33312]: osdmap e761: 8 total, 8 up, 8 in 2024-06-18T13:35:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:07 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3127413520' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]: dispatch 2024-06-18T13:35:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:07 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]: dispatch 2024-06-18T13:35:07.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:07 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:35:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:07 smithi167 ceph-mon[29496]: osdmap e761: 8 total, 8 up, 8 in 2024-06-18T13:35:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:07 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3127413520' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]: dispatch 2024-06-18T13:35:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:07 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]: dispatch 2024-06-18T13:35:07.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:07 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:35:07.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:07 smithi167 ceph-mon[37701]: osdmap e761: 8 total, 8 up, 8 in 2024-06-18T13:35:07.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:07 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3127413520' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]: dispatch 2024-06-18T13:35:07.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:07 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]: dispatch 2024-06-18T13:35:07.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:07 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:35:07.522 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:07.522+0000 7f93f929d640 1 -- 172.21.15.167:0/1270548309 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"} v 0) -- 0x7f93f419a650 con 0x7f93f4100580 2024-06-18T13:35:08.111 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:08.111+0000 7f93ebfff640 1 -- 172.21.15.167:0/1270548309 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a v763) ==== 223+0+0 (secure 0 0 0) 0x7f93e404d020 con 0x7f93f4100580 2024-06-18T13:35:08.222 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:08.222+0000 7f93f929d640 1 -- 172.21.15.167:0/1270548309 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"} v 0) -- 0x7f93f41a6290 con 0x7f93f4100580 2024-06-18T13:35:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:08 smithi179 ceph-mon[33312]: pgmap v1653: 164 pgs: 164 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:08 smithi179 ceph-mon[33312]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:35:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]': finished 2024-06-18T13:35:08.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:08 smithi179 ceph-mon[33312]: osdmap e762: 8 total, 8 up, 8 in 2024-06-18T13:35:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3127413520' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]: dispatch 2024-06-18T13:35:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]: dispatch 2024-06-18T13:35:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:35:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:35:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:08 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:35:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:08 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1270548309' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:35:08.388 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:08 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:35:08.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[37701]: pgmap v1653: 164 pgs: 164 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:08.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[37701]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:35:08.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]': finished 2024-06-18T13:35:08.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[37701]: osdmap e762: 8 total, 8 up, 8 in 2024-06-18T13:35:08.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3127413520' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]: dispatch 2024-06-18T13:35:08.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]: dispatch 2024-06-18T13:35:08.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:35:08.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:35:08.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:35:08.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1270548309' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:35:08.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:35:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[29496]: pgmap v1653: 164 pgs: 164 active+clean; 583 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[29496]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:35:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]': finished 2024-06-18T13:35:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[29496]: osdmap e762: 8 total, 8 up, 8 in 2024-06-18T13:35:08.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3127413520' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]: dispatch 2024-06-18T13:35:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pg_num": 12}]: dispatch 2024-06-18T13:35:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:35:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:35:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:35:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1270548309' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:35:08.479 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:08 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:35:09.132 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.131+0000 7f93ebfff640 1 -- 172.21.15.167:0/1270548309 <== mon.2 v2:172.21.15.167:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a v764) ==== 223+0+0 (secure 0 0 0) 0x7f93e4081640 con 0x7f93f4100580 2024-06-18T13:35:09.132 INFO:tasks.workunit.client.0.smithi167.stderr:set-quota max_objects = 10 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a 2024-06-18T13:35:09.135 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.134+0000 7f93f929d640 1 -- 172.21.15.167:0/1270548309 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f93e006e050 msgr2=0x7f93e0070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:09.135 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.134+0000 7f93f929d640 1 --2- 172.21.15.167:0/1270548309 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f93e006e050 0x7f93e0070510 secure :-1 s=READY pgs=3488 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0095f0 tx=0x7f93dc005f90 comp rx=0 tx=0).stop 2024-06-18T13:35:09.135 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.134+0000 7f93f929d640 1 -- 172.21.15.167:0/1270548309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f4100580 msgr2=0x7f93f4199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:09.135 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.134+0000 7f93f929d640 1 --2- 172.21.15.167:0/1270548309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f4100580 0x7f93f4199300 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f93e4000c00 tx=0x7f93e4004270 comp rx=0 tx=0).stop 2024-06-18T13:35:09.135 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.134+0000 7f93f929d640 1 -- 172.21.15.167:0/1270548309 shutdown_connections 2024-06-18T13:35:09.135 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.134+0000 7f93f929d640 1 --2- 172.21.15.167:0/1270548309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f41051b0 0x7f93f419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:09.135 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.134+0000 7f93f929d640 1 --2- 172.21.15.167:0/1270548309 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f93e006e050 0x7f93e0070510 unknown :-1 s=CLOSED pgs=3488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:09.135 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.134+0000 7f93f929d640 1 --2- 172.21.15.167:0/1270548309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93f4100f20 0x7f93f4199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:09.135 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.134+0000 7f93f929d640 1 --2- 172.21.15.167:0/1270548309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93f4100580 0x7f93f4199300 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:09.135 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.134+0000 7f93f929d640 1 -- 172.21.15.167:0/1270548309 >> 172.21.15.167:0/1270548309 conn(0x7f93f4075f30 msgr2=0x7f93f4068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:35:09.135 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.134+0000 7f93f929d640 1 -- 172.21.15.167:0/1270548309 shutdown_connections 2024-06-18T13:35:09.135 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.135+0000 7f93f929d640 1 -- 172.21.15.167:0/1270548309 wait complete. 2024-06-18T13:35:09.145 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool application enable 2d48a5d0-5bf6-4059-9227-066b6e28e48a rados 2024-06-18T13:35:09.236 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.233+0000 7f9fc0040640 1 -- 172.21.15.167:0/1155551871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb8102d80 msgr2=0x7f9fb810b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:09.236 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.233+0000 7f9fc0040640 1 --2- 172.21.15.167:0/1155551871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb8102d80 0x7f9fb810b260 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009900 tx=0x7f9fa802bcd0 comp rx=0 tx=0).stop 2024-06-18T13:35:09.236 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.234+0000 7f9fc0040640 1 -- 172.21.15.167:0/1155551871 shutdown_connections 2024-06-18T13:35:09.236 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.234+0000 7f9fc0040640 1 --2- 172.21.15.167:0/1155551871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fb810b8b0 0x7f9fb810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:09.236 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.234+0000 7f9fc0040640 1 --2- 172.21.15.167:0/1155551871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb8102d80 0x7f9fb810b260 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:09.237 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.234+0000 7f9fc0040640 1 --2- 172.21.15.167:0/1155551871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb8102460 0x7f9fb8102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:09.237 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.234+0000 7f9fc0040640 1 -- 172.21.15.167:0/1155551871 >> 172.21.15.167:0/1155551871 conn(0x7f9fb80fbe70 msgr2=0x7f9fb80fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:35:09.237 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.234+0000 7f9fc0040640 1 -- 172.21.15.167:0/1155551871 shutdown_connections 2024-06-18T13:35:09.237 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.234+0000 7f9fc0040640 1 -- 172.21.15.167:0/1155551871 wait complete. 2024-06-18T13:35:09.237 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.234+0000 7f9fc0040640 1 Processor -- start 2024-06-18T13:35:09.237 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.234+0000 7f9fc0040640 1 -- start start 2024-06-18T13:35:09.237 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.234+0000 7f9fc0040640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb8102460 0x7f9fb81970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:09.237 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.234+0000 7f9fc0040640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fb8102d80 0x7f9fb8197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:09.237 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.234+0000 7f9fc0040640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb810b8b0 0x7f9fb819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:09.237 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.234+0000 7f9fc0040640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8110b10 con 0x7f9fb8102460 2024-06-18T13:35:09.237 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.234+0000 7f9fc0040640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8110990 con 0x7f9fb810b8b0 2024-06-18T13:35:09.237 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fc0040640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb8110c90 con 0x7f9fb8102d80 2024-06-18T13:35:09.237 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fbdddc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb8102460 0x7f9fb81970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:09.238 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fbd5db640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fb8102d80 0x7f9fb8197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:09.238 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fbdddc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb8102460 0x7f9fb81970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55072/0 (socket says 172.21.15.167:55072) 2024-06-18T13:35:09.238 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fbdddc640 1 -- 172.21.15.167:0/3497180895 learned_addr learned my addr 172.21.15.167:0/3497180895 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:35:09.238 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fbe5dd640 1 --2- 172.21.15.167:0/3497180895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb810b8b0 0x7f9fb819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:09.238 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fbdddc640 1 -- 172.21.15.167:0/3497180895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fb8102d80 msgr2=0x7f9fb8197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:09.238 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fbdddc640 1 --2- 172.21.15.167:0/3497180895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fb8102d80 0x7f9fb8197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:09.238 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fbdddc640 1 -- 172.21.15.167:0/3497180895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb810b8b0 msgr2=0x7f9fb819b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:09.238 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fbdddc640 1 --2- 172.21.15.167:0/3497180895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb810b8b0 0x7f9fb819b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:09.238 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fbdddc640 1 -- 172.21.15.167:0/3497180895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb819c0e0 con 0x7f9fb8102460 2024-06-18T13:35:09.238 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fbe5dd640 1 --2- 172.21.15.167:0/3497180895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb810b8b0 0x7f9fb819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T13:35:09.238 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fbdddc640 1 --2- 172.21.15.167:0/3497180895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb8102460 0x7f9fb81970f0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400c920 tx=0x7f9fb400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:35:09.238 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9faeffd640 1 -- 172.21.15.167:0/3497180895 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4007c70 con 0x7f9fb8102460 2024-06-18T13:35:09.238 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9faeffd640 1 -- 172.21.15.167:0/3497180895 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f9fb4007e10 con 0x7f9fb8102460 2024-06-18T13:35:09.239 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fc0040640 1 -- 172.21.15.167:0/3497180895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb819c3d0 con 0x7f9fb8102460 2024-06-18T13:35:09.239 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9faeffd640 1 -- 172.21.15.167:0/3497180895 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4018690 con 0x7f9fb8102460 2024-06-18T13:35:09.239 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.235+0000 7f9fc0040640 1 -- 172.21.15.167:0/3497180895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb81a3c60 con 0x7f9fb8102460 2024-06-18T13:35:09.239 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.236+0000 7f9fc0040640 1 -- 172.21.15.167:0/3497180895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb8102840 con 0x7f9fb8102460 2024-06-18T13:35:09.240 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.237+0000 7f9faeffd640 1 -- 172.21.15.167:0/3497180895 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f9fb400f450 con 0x7f9fb8102460 2024-06-18T13:35:09.240 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.237+0000 7f9faeffd640 1 --2- 172.21.15.167:0/3497180895 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f9fa406e050 0x7f9fa4070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:09.240 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.238+0000 7f9faeffd640 1 -- 172.21.15.167:0/3497180895 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(764..764 src has 257..764) ==== 9079+0+0 (secure 0 0 0) 0x7f9fb408ee80 con 0x7f9fb8102460 2024-06-18T13:35:09.240 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.238+0000 7f9fbd5db640 1 --2- 172.21.15.167:0/3497180895 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f9fa406e050 0x7f9fa4070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:09.240 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.238+0000 7f9fbd5db640 1 --2- 172.21.15.167:0/3497180895 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f9fa406e050 0x7f9fa4070510 secure :-1 s=READY pgs=3489 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8005ab0 tx=0x7f9fa8005fd0 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:35:09.242 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.242+0000 7f9faeffd640 1 -- 172.21.15.167:0/3497180895 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f9fb4014030 con 0x7f9fb8102460 2024-06-18T13:35:09.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]': finished 2024-06-18T13:35:09.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:09 smithi179 ceph-mon[33312]: osdmap e763: 8 total, 8 up, 8 in 2024-06-18T13:35:09.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:09 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1270548309' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:35:09.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:09 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:35:09.424 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:09.424+0000 7f9fc0040640 1 -- 172.21.15.167:0/3497180895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"} v 0) -- 0x7f9fb81a3e30 con 0x7f9fb8102460 2024-06-18T13:35:09.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]': finished 2024-06-18T13:35:09.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:09 smithi167 ceph-mon[37701]: osdmap e763: 8 total, 8 up, 8 in 2024-06-18T13:35:09.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:09 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1270548309' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:35:09.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:09 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:35:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]': finished 2024-06-18T13:35:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:09 smithi167 ceph-mon[29496]: osdmap e763: 8 total, 8 up, 8 in 2024-06-18T13:35:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:09 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1270548309' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:35:09.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:09 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:35:10.125 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:10.125+0000 7f9faeffd640 1 -- 172.21.15.167:0/3497180895 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"}]=0 enabled application 'rados' on pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' v765) ==== 213+0+0 (secure 0 0 0) 0x7f9fb405bbb0 con 0x7f9fb8102460 2024-06-18T13:35:10.237 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:10.237+0000 7f9fc0040640 1 -- 172.21.15.167:0/3497180895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"} v 0) -- 0x7f9fb81a4080 con 0x7f9fb8102460 2024-06-18T13:35:10.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:35:10 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:35:10.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:10 smithi179 ceph-mon[33312]: pgmap v1656: 176 pgs: 176 active+clean; 583 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:10.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:10 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]': finished 2024-06-18T13:35:10.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:10 smithi179 ceph-mon[33312]: osdmap e764: 8 total, 8 up, 8 in 2024-06-18T13:35:10.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:10 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3497180895' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"}]: dispatch 2024-06-18T13:35:10.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:10 smithi167 ceph-mon[37701]: pgmap v1656: 176 pgs: 176 active+clean; 583 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:10.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:10 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]': finished 2024-06-18T13:35:10.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:10 smithi167 ceph-mon[37701]: osdmap e764: 8 total, 8 up, 8 in 2024-06-18T13:35:10.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:10 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3497180895' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"}]: dispatch 2024-06-18T13:35:10.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:10 smithi167 ceph-mon[29496]: pgmap v1656: 176 pgs: 176 active+clean; 583 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:10.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:10 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "10"}]': finished 2024-06-18T13:35:10.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:10 smithi167 ceph-mon[29496]: osdmap e764: 8 total, 8 up, 8 in 2024-06-18T13:35:10.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:10 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3497180895' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"}]: dispatch 2024-06-18T13:35:11.139 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:11.138+0000 7f9faeffd640 1 -- 172.21.15.167:0/3497180895 <== mon.0 v2:172.21.15.167:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"}]=0 enabled application 'rados' on pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' v766) ==== 213+0+0 (secure 0 0 0) 0x7f9fb405f860 con 0x7f9fb8102460 2024-06-18T13:35:11.139 INFO:tasks.workunit.client.0.smithi167.stderr:enabled application 'rados' on pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' 2024-06-18T13:35:11.142 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:11.141+0000 7f9fc0040640 1 -- 172.21.15.167:0/3497180895 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f9fa406e050 msgr2=0x7f9fa4070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:11.142 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:11.141+0000 7f9fc0040640 1 --2- 172.21.15.167:0/3497180895 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f9fa406e050 0x7f9fa4070510 secure :-1 s=READY pgs=3489 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8005ab0 tx=0x7f9fa8005fd0 comp rx=0 tx=0).stop 2024-06-18T13:35:11.142 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:11.141+0000 7f9fc0040640 1 -- 172.21.15.167:0/3497180895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb8102460 msgr2=0x7f9fb81970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:11.143 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:11.142+0000 7f9fc0040640 1 --2- 172.21.15.167:0/3497180895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb8102460 0x7f9fb81970f0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400c920 tx=0x7f9fb400cde0 comp rx=0 tx=0).stop 2024-06-18T13:35:11.143 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:11.142+0000 7f9fc0040640 1 -- 172.21.15.167:0/3497180895 shutdown_connections 2024-06-18T13:35:11.143 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:11.142+0000 7f9fc0040640 1 --2- 172.21.15.167:0/3497180895 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f9fa406e050 0x7f9fa4070510 unknown :-1 s=CLOSED pgs=3489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:11.143 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:11.142+0000 7f9fc0040640 1 --2- 172.21.15.167:0/3497180895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb810b8b0 0x7f9fb819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:11.143 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:11.142+0000 7f9fc0040640 1 --2- 172.21.15.167:0/3497180895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fb8102d80 0x7f9fb8197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:11.143 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:11.142+0000 7f9fc0040640 1 --2- 172.21.15.167:0/3497180895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb8102460 0x7f9fb81970f0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:11.143 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:11.142+0000 7f9fc0040640 1 -- 172.21.15.167:0/3497180895 >> 172.21.15.167:0/3497180895 conn(0x7f9fb80fbe70 msgr2=0x7f9fb80fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:35:11.143 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:11.142+0000 7f9fc0040640 1 -- 172.21.15.167:0/3497180895 shutdown_connections 2024-06-18T13:35:11.143 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:11.142+0000 7f9fc0040640 1 -- 172.21.15.167:0/3497180895 wait complete. 2024-06-18T13:35:11.153 INFO:tasks.workunit.client.0.smithi167.stderr:++ seq 1 10 2024-06-18T13:35:11.178 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:35:11.178 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put obj1 /etc/passwd 2024-06-18T13:35:11.214 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:35:11.215 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put obj2 /etc/passwd 2024-06-18T13:35:11.251 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:35:11.251 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put obj3 /etc/passwd 2024-06-18T13:35:11.286 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:35:11.286 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put obj4 /etc/passwd 2024-06-18T13:35:11.321 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:35:11.321 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put obj5 /etc/passwd 2024-06-18T13:35:11.358 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:35:11.359 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put obj6 /etc/passwd 2024-06-18T13:35:11.394 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:35:11.394 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put obj7 /etc/passwd 2024-06-18T13:35:11.429 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:35:11.430 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put obj8 /etc/passwd 2024-06-18T13:35:11.465 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:35:11.465 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put obj9 /etc/passwd 2024-06-18T13:35:11.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3497180895' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"}]': finished 2024-06-18T13:35:11.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:11 smithi167 ceph-mon[29496]: osdmap e765: 8 total, 8 up, 8 in 2024-06-18T13:35:11.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:11 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3497180895' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"}]: dispatch 2024-06-18T13:35:11.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:11 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:11.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:35:11.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3497180895' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"}]': finished 2024-06-18T13:35:11.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:11 smithi167 ceph-mon[37701]: osdmap e765: 8 total, 8 up, 8 in 2024-06-18T13:35:11.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:11 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3497180895' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"}]: dispatch 2024-06-18T13:35:11.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:11 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:11.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:35:11.500 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:35:11.500 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put obj10 /etc/passwd 2024-06-18T13:35:11.536 INFO:tasks.workunit.client.0.smithi167.stderr:+ sleep 30 2024-06-18T13:35:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3497180895' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"}]': finished 2024-06-18T13:35:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:11 smithi179 ceph-mon[33312]: osdmap e765: 8 total, 8 up, 8 in 2024-06-18T13:35:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:11 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3497180895' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"}]: dispatch 2024-06-18T13:35:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:11 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:11.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:35:12.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:12 smithi167 ceph-mon[37701]: pgmap v1659: 176 pgs: 176 active+clean; 583 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:12.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:12 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3497180895' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"}]': finished 2024-06-18T13:35:12.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:12 smithi167 ceph-mon[37701]: osdmap e766: 8 total, 8 up, 8 in 2024-06-18T13:35:12.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:12 smithi167 ceph-mon[29496]: pgmap v1659: 176 pgs: 176 active+clean; 583 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:12.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:12 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3497180895' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"}]': finished 2024-06-18T13:35:12.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:12 smithi167 ceph-mon[29496]: osdmap e766: 8 total, 8 up, 8 in 2024-06-18T13:35:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:12 smithi179 ceph-mon[33312]: pgmap v1659: 176 pgs: 176 active+clean; 583 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:12 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3497180895' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "app": "rados"}]': finished 2024-06-18T13:35:12.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:12 smithi179 ceph-mon[33312]: osdmap e766: 8 total, 8 up, 8 in 2024-06-18T13:35:14.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:14 smithi167 ceph-mon[37701]: pgmap v1661: 176 pgs: 176 active+clean; 583 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:35:14.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:14 smithi167 ceph-mon[29496]: pgmap v1661: 176 pgs: 176 active+clean; 583 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:35:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:14 smithi179 ceph-mon[33312]: pgmap v1661: 176 pgs: 176 active+clean; 583 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-18T13:35:15.379 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:35:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:35:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:35:16.397 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:16 smithi167 ceph-mon[37701]: pgmap v1662: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 20 KiB/s wr, 2 op/s 2024-06-18T13:35:16.398 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:16 smithi167 ceph-mon[29496]: pgmap v1662: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 20 KiB/s wr, 2 op/s 2024-06-18T13:35:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:16 smithi179 ceph-mon[33312]: pgmap v1662: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 20 KiB/s wr, 2 op/s 2024-06-18T13:35:18.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:18 smithi167 ceph-mon[37701]: pgmap v1663: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 675 B/s rd, 16 KiB/s wr, 1 op/s 2024-06-18T13:35:18.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:18 smithi167 ceph-mon[29496]: pgmap v1663: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 675 B/s rd, 16 KiB/s wr, 1 op/s 2024-06-18T13:35:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:18 smithi179 ceph-mon[33312]: pgmap v1663: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 675 B/s rd, 16 KiB/s wr, 1 op/s 2024-06-18T13:35:19.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:19 smithi167 ceph-mon[37701]: pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' is full (reached quota's max_objects: 10) 2024-06-18T13:35:19.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:19 smithi167 ceph-mon[37701]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-18T13:35:19.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:19 smithi167 ceph-mon[37701]: osdmap e767: 8 total, 8 up, 8 in 2024-06-18T13:35:19.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:19 smithi167 ceph-mon[29496]: pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' is full (reached quota's max_objects: 10) 2024-06-18T13:35:19.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:19 smithi167 ceph-mon[29496]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-18T13:35:19.478 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:19 smithi167 ceph-mon[29496]: osdmap e767: 8 total, 8 up, 8 in 2024-06-18T13:35:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:19 smithi179 ceph-mon[33312]: pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' is full (reached quota's max_objects: 10) 2024-06-18T13:35:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:19 smithi179 ceph-mon[33312]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-18T13:35:19.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:19 smithi179 ceph-mon[33312]: osdmap e767: 8 total, 8 up, 8 in 2024-06-18T13:35:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:20 smithi179 ceph-mon[33312]: pgmap v1665: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-06-18T13:35:20.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:35:20 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:35:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:20 smithi167 ceph-mon[29496]: pgmap v1665: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-06-18T13:35:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:20 smithi167 ceph-mon[37701]: pgmap v1665: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-06-18T13:35:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:21 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:21.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:21 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:21 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:22 smithi167 ceph-mon[37701]: pgmap v1666: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-06-18T13:35:22.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:22 smithi167 ceph-mon[29496]: pgmap v1666: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-06-18T13:35:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:22 smithi179 ceph-mon[33312]: pgmap v1666: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-06-18T13:35:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:24 smithi167 ceph-mon[37701]: pgmap v1667: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-18T13:35:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:24 smithi167 ceph-mon[29496]: pgmap v1667: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-18T13:35:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:24 smithi179 ceph-mon[33312]: pgmap v1667: 176 pgs: 176 active+clean; 701 KiB data, 891 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-18T13:35:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:35:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:35:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:35:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:26 smithi167 ceph-mon[37701]: pgmap v1668: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:35:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:26 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:35:26.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:26 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:35:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:26 smithi167 ceph-mon[29496]: pgmap v1668: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:35:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:26 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:35:26.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:26 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:35:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:26 smithi179 ceph-mon[33312]: pgmap v1668: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:35:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:26 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:35:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:26 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:35:28.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:28 smithi167 ceph-mon[37701]: pgmap v1669: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:35:28.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:28 smithi167 ceph-mon[29496]: pgmap v1669: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:35:28.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:28 smithi179 ceph-mon[33312]: pgmap v1669: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:35:30.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:30 smithi179 ceph-mon[33312]: pgmap v1670: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:35:30.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:35:30 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:35:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:30 smithi167 ceph-mon[37701]: pgmap v1670: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:35:30.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:30 smithi167 ceph-mon[29496]: pgmap v1670: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-18T13:35:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:31 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:31.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:31 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:31 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:32.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:32 smithi167 ceph-mon[37701]: pgmap v1671: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:35:32.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:32 smithi167 ceph-mon[29496]: pgmap v1671: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:35:32.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:32 smithi179 ceph-mon[33312]: pgmap v1671: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:35:34.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:34 smithi167 ceph-mon[37701]: pgmap v1672: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:35:34.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:34 smithi167 ceph-mon[29496]: pgmap v1672: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:35:34.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:34 smithi179 ceph-mon[33312]: pgmap v1672: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:35:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:35:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:35:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:35:36.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:36 smithi167 ceph-mon[37701]: pgmap v1673: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:36.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:36 smithi167 ceph-mon[29496]: pgmap v1673: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:36.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:36 smithi179 ceph-mon[33312]: pgmap v1673: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:38 smithi179 ceph-mon[33312]: pgmap v1674: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:35:38.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:38 smithi167 ceph-mon[37701]: pgmap v1674: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:35:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:38 smithi167 ceph-mon[29496]: pgmap v1674: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:35:40.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:40 smithi179 ceph-mon[33312]: pgmap v1675: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:40.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:35:40 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:35:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:40 smithi167 ceph-mon[37701]: pgmap v1675: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:40 smithi167 ceph-mon[29496]: pgmap v1675: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:41.537 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=150083 2024-06-18T13:35:41.537 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool set-quota 2d48a5d0-5bf6-4059-9227-066b6e28e48a max_objects 100 2024-06-18T13:35:41.537 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put onemore /etc/passwd 2024-06-18T13:35:41.634 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.632+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2770666529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc2010b8b0 msgr2=0x7fbc2010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:41.634 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.632+0000 7fbc27ac2640 1 --2- 172.21.15.167:0/2770666529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc2010b8b0 0x7fbc2010dd30 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009be0 tx=0x7fbc1802bf80 comp rx=0 tx=0).stop 2024-06-18T13:35:41.634 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.632+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2770666529 shutdown_connections 2024-06-18T13:35:41.634 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.632+0000 7fbc27ac2640 1 --2- 172.21.15.167:0/2770666529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc2010b8b0 0x7fbc2010dd30 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:41.634 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.632+0000 7fbc27ac2640 1 --2- 172.21.15.167:0/2770666529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc20102d80 0x7fbc2010b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:41.634 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.632+0000 7fbc27ac2640 1 --2- 172.21.15.167:0/2770666529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc20102460 0x7fbc20102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:41.635 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.632+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2770666529 >> 172.21.15.167:0/2770666529 conn(0x7fbc200fbe90 msgr2=0x7fbc200fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:35:41.635 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.632+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2770666529 shutdown_connections 2024-06-18T13:35:41.635 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.632+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2770666529 wait complete. 2024-06-18T13:35:41.635 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.632+0000 7fbc27ac2640 1 Processor -- start 2024-06-18T13:35:41.635 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.632+0000 7fbc27ac2640 1 -- start start 2024-06-18T13:35:41.635 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc27ac2640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc20102460 0x7fbc201970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:41.635 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc27ac2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc20102d80 0x7fbc20197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:41.635 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc27ac2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc2010b8b0 0x7fbc2019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:41.635 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc27ac2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbc20110ad0 con 0x7fbc20102d80 2024-06-18T13:35:41.635 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc27ac2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbc20110950 con 0x7fbc2010b8b0 2024-06-18T13:35:41.635 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc27ac2640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbc20110c50 con 0x7fbc20102460 2024-06-18T13:35:41.635 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc2585e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc20102460 0x7fbc201970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:41.635 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc2505d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc20102d80 0x7fbc20197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:41.635 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc2585e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc20102460 0x7fbc201970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:45496/0 (socket says 172.21.15.167:45496) 2024-06-18T13:35:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc2585e640 1 -- 172.21.15.167:0/2256580365 learned_addr learned my addr 172.21.15.167:0/2256580365 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:35:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc2605f640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc2010b8b0 0x7fbc2019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc2585e640 1 -- 172.21.15.167:0/2256580365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc2010b8b0 msgr2=0x7fbc2019b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc2585e640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc2010b8b0 0x7fbc2019b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc2585e640 1 -- 172.21.15.167:0/2256580365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc20102d80 msgr2=0x7fbc20197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc2585e640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc20102d80 0x7fbc20197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc2585e640 1 -- 172.21.15.167:0/2256580365 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2019c0e0 con 0x7fbc20102460 2024-06-18T13:35:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc2505d640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc20102d80 0x7fbc20197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:35:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc2605f640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc2010b8b0 0x7fbc2019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-18T13:35:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc2585e640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc20102460 0x7fbc201970f0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400e9c0 tx=0x7fbc1400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:35:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc12ffd640 1 -- 172.21.15.167:0/2256580365 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1400cde0 con 0x7fbc20102460 2024-06-18T13:35:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc12ffd640 1 -- 172.21.15.167:0/2256580365 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fbc140035f0 con 0x7fbc20102460 2024-06-18T13:35:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2256580365 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2019c3d0 con 0x7fbc20102460 2024-06-18T13:35:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc12ffd640 1 -- 172.21.15.167:0/2256580365 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc140044a0 con 0x7fbc20102460 2024-06-18T13:35:41.637 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.633+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2256580365 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc201a3c60 con 0x7fbc20102460 2024-06-18T13:35:41.637 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.634+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2256580365 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc20102840 con 0x7fbc20102460 2024-06-18T13:35:41.638 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.635+0000 7fbc12ffd640 1 -- 172.21.15.167:0/2256580365 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7fbc14004000 con 0x7fbc20102460 2024-06-18T13:35:41.638 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.635+0000 7fbc12ffd640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbc0c06e050 0x7fbc0c070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:41.638 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.635+0000 7fbc12ffd640 1 -- 172.21.15.167:0/2256580365 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(767..767 src has 257..767) ==== 9092+0+0 (secure 0 0 0) 0x7fbc1408f090 con 0x7fbc20102460 2024-06-18T13:35:41.638 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.636+0000 7fbc2505d640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbc0c06e050 0x7fbc0c070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:41.638 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.636+0000 7fbc12ffd640 1 -- 172.21.15.167:0/2256580365 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=768}) -- 0x7fbc0c079b50 con 0x7fbc20102460 2024-06-18T13:35:41.638 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.636+0000 7fbc2505d640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbc0c06e050 0x7fbc0c070510 secure :-1 s=READY pgs=3501 cs=0 l=1 rev1=1 crypto rx=0x7fbc20106250 tx=0x7fbc1c005f90 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:35:41.640 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.640+0000 7fbc12ffd640 1 -- 172.21.15.167:0/2256580365 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fbc1405bdb0 con 0x7fbc20102460 2024-06-18T13:35:41.814 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:41.815+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2256580365 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"} v 0) -- 0x7fbc201a3e90 con 0x7fbc20102460 2024-06-18T13:35:41.886 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:41 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:41 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:35:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:41 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:41 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:35:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:41 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:41 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:35:42.494 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:42.493+0000 7fbc12ffd640 1 -- 172.21.15.167:0/2256580365 <== mon.2 v2:172.21.15.167:3301/0 7 ==== osd_map(768..768 src has 257..768) ==== 628+0+0 (secure 0 0 0) 0x7fbc14059750 con 0x7fbc20102460 2024-06-18T13:35:42.494 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:42.493+0000 7fbc12ffd640 1 -- 172.21.15.167:0/2256580365 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=769}) -- 0x7fbc0c07a6c0 con 0x7fbc20102460 2024-06-18T13:35:42.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:42.497+0000 7fbc12ffd640 1 -- 172.21.15.167:0/2256580365 <== mon.2 v2:172.21.15.167:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a v768) ==== 225+0+0 (secure 0 0 0) 0x7fbc1405fa60 con 0x7fbc20102460 2024-06-18T13:35:42.607 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:42.607+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2256580365 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"} v 0) -- 0x7fbc201a40e0 con 0x7fbc20102460 2024-06-18T13:35:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:42 smithi179 ceph-mon[33312]: pgmap v1676: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:35:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:42 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2256580365' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]: dispatch 2024-06-18T13:35:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:42 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]: dispatch 2024-06-18T13:35:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:42 smithi167 ceph-mon[29496]: pgmap v1676: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:35:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:42 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2256580365' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]: dispatch 2024-06-18T13:35:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:42 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]: dispatch 2024-06-18T13:35:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:42 smithi167 ceph-mon[37701]: pgmap v1676: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:35:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:42 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2256580365' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]: dispatch 2024-06-18T13:35:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:42 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]: dispatch 2024-06-18T13:35:43.445 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.445+0000 7fbc12ffd640 1 -- 172.21.15.167:0/2256580365 <== mon.2 v2:172.21.15.167:3301/0 9 ==== osd_map(769..769 src has 257..769) ==== 628+0+0 (secure 0 0 0) 0x7fbc1408c910 con 0x7fbc20102460 2024-06-18T13:35:43.446 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.445+0000 7fbc12ffd640 1 -- 172.21.15.167:0/2256580365 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=770}) -- 0x7fbc0c07acc0 con 0x7fbc20102460 2024-06-18T13:35:43.450 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.449+0000 7fbc12ffd640 1 -- 172.21.15.167:0/2256580365 <== mon.2 v2:172.21.15.167:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a v769) ==== 225+0+0 (secure 0 0 0) 0x7fbc14064290 con 0x7fbc20102460 2024-06-18T13:35:43.450 INFO:tasks.workunit.client.0.smithi167.stderr:set-quota max_objects = 100 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a 2024-06-18T13:35:43.454 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.454+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2256580365 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbc0c06e050 msgr2=0x7fbc0c070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:43.454 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.454+0000 7fbc27ac2640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbc0c06e050 0x7fbc0c070510 secure :-1 s=READY pgs=3501 cs=0 l=1 rev1=1 crypto rx=0x7fbc20106250 tx=0x7fbc1c005f90 comp rx=0 tx=0).stop 2024-06-18T13:35:43.454 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.454+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2256580365 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc20102460 msgr2=0x7fbc201970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:43.454 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.454+0000 7fbc27ac2640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc20102460 0x7fbc201970f0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400e9c0 tx=0x7fbc1400ee80 comp rx=0 tx=0).stop 2024-06-18T13:35:43.454 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.454+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2256580365 shutdown_connections 2024-06-18T13:35:43.454 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.454+0000 7fbc27ac2640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbc0c06e050 0x7fbc0c070510 unknown :-1 s=CLOSED pgs=3501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:43.454 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.454+0000 7fbc27ac2640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc2010b8b0 0x7fbc2019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:43.455 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.454+0000 7fbc27ac2640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc20102d80 0x7fbc20197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:43.455 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.454+0000 7fbc27ac2640 1 --2- 172.21.15.167:0/2256580365 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc20102460 0x7fbc201970f0 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:43.455 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.454+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2256580365 >> 172.21.15.167:0/2256580365 conn(0x7fbc200fbe90 msgr2=0x7fbc201031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:35:43.455 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.454+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2256580365 shutdown_connections 2024-06-18T13:35:43.455 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.454+0000 7fbc27ac2640 1 -- 172.21.15.167:0/2256580365 wait complete. 2024-06-18T13:35:43.465 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 150083 2024-06-18T13:35:43.465 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 0 -ne 0 ']' 2024-06-18T13:35:43.465 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2024-06-18T13:35:43.465 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put twomore /etc/passwd 2024-06-18T13:35:43.503 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool set-quota 2d48a5d0-5bf6-4059-9227-066b6e28e48a max_bytes 100 2024-06-18T13:35:43.590 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.588+0000 7fbc52b4f640 1 -- 172.21.15.167:0/4252133171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc4c105280 msgr2=0x7fbc4c105660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:43.590 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.588+0000 7fbc52b4f640 1 --2- 172.21.15.167:0/4252133171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc4c105280 0x7fbc4c105660 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009860 tx=0x7fbc4002bc00 comp rx=0 tx=0).stop 2024-06-18T13:35:43.590 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.588+0000 7fbc52b4f640 1 -- 172.21.15.167:0/4252133171 shutdown_connections 2024-06-18T13:35:43.591 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.588+0000 7fbc52b4f640 1 --2- 172.21.15.167:0/4252133171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc4c1017e0 0x7fbc4c10cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:43.591 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.588+0000 7fbc52b4f640 1 --2- 172.21.15.167:0/4252133171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc4c100e60 0x7fbc4c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:43.591 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.588+0000 7fbc52b4f640 1 --2- 172.21.15.167:0/4252133171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc4c105280 0x7fbc4c105660 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:43.591 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.588+0000 7fbc52b4f640 1 -- 172.21.15.167:0/4252133171 >> 172.21.15.167:0/4252133171 conn(0x7fbc4c0756d0 msgr2=0x7fbc4c075ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:35:43.591 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.588+0000 7fbc52b4f640 1 -- 172.21.15.167:0/4252133171 shutdown_connections 2024-06-18T13:35:43.591 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.588+0000 7fbc52b4f640 1 -- 172.21.15.167:0/4252133171 wait complete. 2024-06-18T13:35:43.591 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.588+0000 7fbc52b4f640 1 Processor -- start 2024-06-18T13:35:43.591 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.588+0000 7fbc52b4f640 1 -- start start 2024-06-18T13:35:43.591 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc52b4f640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc4c100e60 0x7fbc4c102020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:43.591 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc52b4f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc4c1017e0 0x7fbc4c102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:43.591 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc52b4f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc4c105280 0x7fbc4c102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:43.591 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc52b4f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c10f990 con 0x7fbc4c1017e0 2024-06-18T13:35:43.592 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc52b4f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c10f810 con 0x7fbc4c105280 2024-06-18T13:35:43.592 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc52b4f640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbc4c10fb10 con 0x7fbc4c100e60 2024-06-18T13:35:43.592 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc4bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc4c1017e0 0x7fbc4c102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:43.592 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc508eb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc4c100e60 0x7fbc4c102020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:43.592 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc4bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc4c1017e0 0x7fbc4c102560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38728/0 (socket says 172.21.15.167:38728) 2024-06-18T13:35:43.592 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc4bfff640 1 -- 172.21.15.167:0/908014826 learned_addr learned my addr 172.21.15.167:0/908014826 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:35:43.592 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc508eb640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc4c100e60 0x7fbc4c102020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:55142/0 (socket says 172.21.15.167:55142) 2024-06-18T13:35:43.592 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc510ec640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc4c105280 0x7fbc4c102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:43.592 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc4bfff640 1 -- 172.21.15.167:0/908014826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc4c100e60 msgr2=0x7fbc4c102020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:43.592 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc4bfff640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc4c100e60 0x7fbc4c102020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:43.592 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc4bfff640 1 -- 172.21.15.167:0/908014826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc4c105280 msgr2=0x7fbc4c102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:43.592 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc4bfff640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc4c105280 0x7fbc4c102aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:43.592 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc4bfff640 1 -- 172.21.15.167:0/908014826 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4c1a55a0 con 0x7fbc4c1017e0 2024-06-18T13:35:43.593 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc508eb640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc4c100e60 0x7fbc4c102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:35:43.593 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc510ec640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc4c105280 0x7fbc4c102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T13:35:43.593 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc4bfff640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc4c1017e0 0x7fbc4c102560 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800d950 tx=0x7fbc3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:35:43.593 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc49ffb640 1 -- 172.21.15.167:0/908014826 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38017070 con 0x7fbc4c1017e0 2024-06-18T13:35:43.593 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc49ffb640 1 -- 172.21.15.167:0/908014826 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fbc38009c30 con 0x7fbc4c1017e0 2024-06-18T13:35:43.593 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc49ffb640 1 -- 172.21.15.167:0/908014826 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3800f620 con 0x7fbc4c1017e0 2024-06-18T13:35:43.593 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.589+0000 7fbc52b4f640 1 -- 172.21.15.167:0/908014826 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4c1a5890 con 0x7fbc4c1017e0 2024-06-18T13:35:43.593 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.590+0000 7fbc52b4f640 1 -- 172.21.15.167:0/908014826 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc4c1a5de0 con 0x7fbc4c1017e0 2024-06-18T13:35:43.594 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.591+0000 7fbc52b4f640 1 -- 172.21.15.167:0/908014826 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc4c1012a0 con 0x7fbc4c1017e0 2024-06-18T13:35:43.594 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.591+0000 7fbc49ffb640 1 -- 172.21.15.167:0/908014826 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7fbc38010460 con 0x7fbc4c1017e0 2024-06-18T13:35:43.594 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.592+0000 7fbc49ffb640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbc3c06e050 0x7fbc3c070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:35:43.594 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.592+0000 7fbc508eb640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbc3c06e050 0x7fbc3c070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:35:43.594 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.592+0000 7fbc49ffb640 1 -- 172.21.15.167:0/908014826 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(769..769 src has 257..769) ==== 9092+0+0 (secure 0 0 0) 0x7fbc3808e8f0 con 0x7fbc4c1017e0 2024-06-18T13:35:43.595 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.592+0000 7fbc508eb640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbc3c06e050 0x7fbc3c070510 secure :-1 s=READY pgs=3503 cs=0 l=1 rev1=1 crypto rx=0x7fbc4002c210 tx=0x7fbc400058d0 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:35:43.596 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.596+0000 7fbc49ffb640 1 -- 172.21.15.167:0/908014826 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fbc3805f2c0 con 0x7fbc4c1017e0 2024-06-18T13:35:43.772 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:43.772+0000 7fbc52b4f640 1 -- 172.21.15.167:0/908014826 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"} v 0) -- 0x7fbc4c1096a0 con 0x7fbc4c1017e0 2024-06-18T13:35:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]': finished 2024-06-18T13:35:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:43 smithi179 ceph-mon[33312]: osdmap e768: 8 total, 8 up, 8 in 2024-06-18T13:35:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:43 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2256580365' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]: dispatch 2024-06-18T13:35:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]: dispatch 2024-06-18T13:35:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:43 smithi179 ceph-mon[33312]: pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' no longer out of quota; removing NO_QUOTA flag 2024-06-18T13:35:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:43 smithi179 ceph-mon[33312]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-18T13:35:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:43 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]': finished 2024-06-18T13:35:43.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:43 smithi179 ceph-mon[33312]: osdmap e769: 8 total, 8 up, 8 in 2024-06-18T13:35:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]': finished 2024-06-18T13:35:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[37701]: osdmap e768: 8 total, 8 up, 8 in 2024-06-18T13:35:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2256580365' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]: dispatch 2024-06-18T13:35:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]: dispatch 2024-06-18T13:35:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[37701]: pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' no longer out of quota; removing NO_QUOTA flag 2024-06-18T13:35:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[37701]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-18T13:35:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]': finished 2024-06-18T13:35:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[37701]: osdmap e769: 8 total, 8 up, 8 in 2024-06-18T13:35:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]': finished 2024-06-18T13:35:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[29496]: osdmap e768: 8 total, 8 up, 8 in 2024-06-18T13:35:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2256580365' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]: dispatch 2024-06-18T13:35:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]: dispatch 2024-06-18T13:35:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[29496]: pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' no longer out of quota; removing NO_QUOTA flag 2024-06-18T13:35:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[29496]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-18T13:35:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "100"}]': finished 2024-06-18T13:35:43.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:43 smithi167 ceph-mon[29496]: osdmap e769: 8 total, 8 up, 8 in 2024-06-18T13:35:44.513 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:44.513+0000 7fbc49ffb640 1 -- 172.21.15.167:0/908014826 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a v770) ==== 221+0+0 (secure 0 0 0) 0x7fbc38063af0 con 0x7fbc4c1017e0 2024-06-18T13:35:44.627 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:44.627+0000 7fbc52b4f640 1 -- 172.21.15.167:0/908014826 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"} v 0) -- 0x7fbc4c1092b0 con 0x7fbc4c1017e0 2024-06-18T13:35:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:44 smithi179 ceph-mon[33312]: pgmap v1678: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:35:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:44 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/908014826' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"}]: dispatch 2024-06-18T13:35:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:44 smithi167 ceph-mon[37701]: pgmap v1678: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:35:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:44 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/908014826' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"}]: dispatch 2024-06-18T13:35:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:44 smithi167 ceph-mon[29496]: pgmap v1678: 176 pgs: 176 active+clean; 701 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:35:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:44 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/908014826' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"}]: dispatch 2024-06-18T13:35:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:35:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:35:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:35:45.531 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:45.531+0000 7fbc49ffb640 1 -- 172.21.15.167:0/908014826 <== mon.0 v2:172.21.15.167:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a v771) ==== 221+0+0 (secure 0 0 0) 0x7fbc38059060 con 0x7fbc4c1017e0 2024-06-18T13:35:45.531 INFO:tasks.workunit.client.0.smithi167.stderr:set-quota max_bytes = 100 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a 2024-06-18T13:35:45.534 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:45.534+0000 7fbc52b4f640 1 -- 172.21.15.167:0/908014826 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbc3c06e050 msgr2=0x7fbc3c070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:45.534 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:45.534+0000 7fbc52b4f640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbc3c06e050 0x7fbc3c070510 secure :-1 s=READY pgs=3503 cs=0 l=1 rev1=1 crypto rx=0x7fbc4002c210 tx=0x7fbc400058d0 comp rx=0 tx=0).stop 2024-06-18T13:35:45.534 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:45.534+0000 7fbc52b4f640 1 -- 172.21.15.167:0/908014826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc4c1017e0 msgr2=0x7fbc4c102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:35:45.535 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:45.534+0000 7fbc52b4f640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc4c1017e0 0x7fbc4c102560 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800d950 tx=0x7fbc3800de10 comp rx=0 tx=0).stop 2024-06-18T13:35:45.535 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:45.534+0000 7fbc52b4f640 1 -- 172.21.15.167:0/908014826 shutdown_connections 2024-06-18T13:35:45.535 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:45.534+0000 7fbc52b4f640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbc3c06e050 0x7fbc3c070510 unknown :-1 s=CLOSED pgs=3503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:45.535 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:45.534+0000 7fbc52b4f640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc4c105280 0x7fbc4c102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:45.535 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:45.534+0000 7fbc52b4f640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc4c1017e0 0x7fbc4c102560 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:45.535 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:45.534+0000 7fbc52b4f640 1 --2- 172.21.15.167:0/908014826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc4c100e60 0x7fbc4c102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:35:45.535 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:45.534+0000 7fbc52b4f640 1 -- 172.21.15.167:0/908014826 >> 172.21.15.167:0/908014826 conn(0x7fbc4c0756d0 msgr2=0x7fbc4c104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:35:45.535 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:45.535+0000 7fbc52b4f640 1 -- 172.21.15.167:0/908014826 shutdown_connections 2024-06-18T13:35:45.536 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:35:45.535+0000 7fbc52b4f640 1 -- 172.21.15.167:0/908014826 wait complete. 2024-06-18T13:35:45.549 INFO:tasks.workunit.client.0.smithi167.stderr:+ sleep 30 2024-06-18T13:35:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/908014826' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"}]': finished 2024-06-18T13:35:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:45 smithi179 ceph-mon[33312]: osdmap e770: 8 total, 8 up, 8 in 2024-06-18T13:35:45.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:45 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/908014826' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"}]: dispatch 2024-06-18T13:35:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/908014826' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"}]': finished 2024-06-18T13:35:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:45 smithi167 ceph-mon[29496]: osdmap e770: 8 total, 8 up, 8 in 2024-06-18T13:35:45.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:45 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/908014826' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"}]: dispatch 2024-06-18T13:35:45.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/908014826' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"}]': finished 2024-06-18T13:35:45.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:45 smithi167 ceph-mon[37701]: osdmap e770: 8 total, 8 up, 8 in 2024-06-18T13:35:45.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:45 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/908014826' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"}]: dispatch 2024-06-18T13:35:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:46 smithi179 ceph-mon[33312]: pgmap v1681: 176 pgs: 176 active+clean; 701 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:35:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:46 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/908014826' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"}]': finished 2024-06-18T13:35:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:46 smithi179 ceph-mon[33312]: osdmap e771: 8 total, 8 up, 8 in 2024-06-18T13:35:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:46 smithi167 ceph-mon[29496]: pgmap v1681: 176 pgs: 176 active+clean; 701 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:35:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:46 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/908014826' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"}]': finished 2024-06-18T13:35:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:46 smithi167 ceph-mon[29496]: osdmap e771: 8 total, 8 up, 8 in 2024-06-18T13:35:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:46 smithi167 ceph-mon[37701]: pgmap v1681: 176 pgs: 176 active+clean; 701 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:35:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:46 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/908014826' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "100"}]': finished 2024-06-18T13:35:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:46 smithi167 ceph-mon[37701]: osdmap e771: 8 total, 8 up, 8 in 2024-06-18T13:35:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:48 smithi179 ceph-mon[33312]: pgmap v1683: 176 pgs: 176 active+clean; 701 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:48 smithi179 ceph-mon[33312]: pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' is full (reached quota's max_bytes: 100 B) 2024-06-18T13:35:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:48 smithi179 ceph-mon[33312]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-18T13:35:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:48 smithi179 ceph-mon[33312]: osdmap e772: 8 total, 8 up, 8 in 2024-06-18T13:35:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:48 smithi167 ceph-mon[37701]: pgmap v1683: 176 pgs: 176 active+clean; 701 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:48 smithi167 ceph-mon[37701]: pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' is full (reached quota's max_bytes: 100 B) 2024-06-18T13:35:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:48 smithi167 ceph-mon[37701]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-18T13:35:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:48 smithi167 ceph-mon[37701]: osdmap e772: 8 total, 8 up, 8 in 2024-06-18T13:35:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:48 smithi167 ceph-mon[29496]: pgmap v1683: 176 pgs: 176 active+clean; 701 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:35:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:48 smithi167 ceph-mon[29496]: pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' is full (reached quota's max_bytes: 100 B) 2024-06-18T13:35:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:48 smithi167 ceph-mon[29496]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-18T13:35:48.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:48 smithi167 ceph-mon[29496]: osdmap e772: 8 total, 8 up, 8 in 2024-06-18T13:35:50.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:50 smithi179 ceph-mon[33312]: pgmap v1685: 176 pgs: 176 active+clean; 724 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 4.6 KiB/s wr, 2 op/s 2024-06-18T13:35:50.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:35:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:35:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:50 smithi167 ceph-mon[37701]: pgmap v1685: 176 pgs: 176 active+clean; 724 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 4.6 KiB/s wr, 2 op/s 2024-06-18T13:35:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:50 smithi167 ceph-mon[29496]: pgmap v1685: 176 pgs: 176 active+clean; 724 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 4.6 KiB/s wr, 2 op/s 2024-06-18T13:35:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:51 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:51 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:51 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:35:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:52 smithi179 ceph-mon[33312]: pgmap v1686: 176 pgs: 176 active+clean; 724 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 825 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-06-18T13:35:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:52 smithi167 ceph-mon[37701]: pgmap v1686: 176 pgs: 176 active+clean; 724 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 825 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-06-18T13:35:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:52 smithi167 ceph-mon[29496]: pgmap v1686: 176 pgs: 176 active+clean; 724 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 825 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-06-18T13:35:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:54 smithi179 ceph-mon[33312]: pgmap v1687: 176 pgs: 176 active+clean; 724 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 3.0 KiB/s wr, 0 op/s 2024-06-18T13:35:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:54 smithi167 ceph-mon[37701]: pgmap v1687: 176 pgs: 176 active+clean; 724 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 3.0 KiB/s wr, 0 op/s 2024-06-18T13:35:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:54 smithi167 ceph-mon[29496]: pgmap v1687: 176 pgs: 176 active+clean; 724 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 3.0 KiB/s wr, 0 op/s 2024-06-18T13:35:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:35:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:35:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:35:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:56 smithi179 ceph-mon[33312]: pgmap v1688: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-06-18T13:35:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:35:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:56 smithi167 ceph-mon[37701]: pgmap v1688: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-06-18T13:35:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:35:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:56 smithi167 ceph-mon[29496]: pgmap v1688: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-06-18T13:35:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:35:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:35:58 smithi179 ceph-mon[33312]: pgmap v1689: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-06-18T13:35:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:35:58 smithi167 ceph-mon[37701]: pgmap v1689: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-06-18T13:35:58.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:35:58 smithi167 ceph-mon[29496]: pgmap v1689: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-06-18T13:36:00.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:00 smithi179 ceph-mon[33312]: pgmap v1690: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.3 KiB/s wr, 1 op/s 2024-06-18T13:36:00.638 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:36:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:36:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:00 smithi167 ceph-mon[37701]: pgmap v1690: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.3 KiB/s wr, 1 op/s 2024-06-18T13:36:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:00 smithi167 ceph-mon[29496]: pgmap v1690: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.3 KiB/s wr, 1 op/s 2024-06-18T13:36:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:01 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:01 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:01.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:01 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:02.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:02 smithi179 ceph-mon[33312]: pgmap v1691: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:02.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:02 smithi167 ceph-mon[29496]: pgmap v1691: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:02.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:02 smithi167 ceph-mon[37701]: pgmap v1691: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:04.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:04 smithi179 ceph-mon[33312]: pgmap v1692: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:04.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:04 smithi167 ceph-mon[37701]: pgmap v1692: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:04.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:04 smithi167 ceph-mon[29496]: pgmap v1692: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:36:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:36:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:36:06.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:06 smithi179 ceph-mon[33312]: pgmap v1693: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:06.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:06 smithi167 ceph-mon[37701]: pgmap v1693: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:06.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:06 smithi167 ceph-mon[29496]: pgmap v1693: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:07.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:07 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:36:07.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:07 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:36:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:07 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:36:08.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:08 smithi179 ceph-mon[33312]: pgmap v1694: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:08.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:08 smithi167 ceph-mon[37701]: pgmap v1694: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:08.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:08 smithi167 ceph-mon[29496]: pgmap v1694: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:10.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:36:10 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:36:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:10 smithi179 ceph-mon[33312]: pgmap v1695: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:10.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:10 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:10 smithi167 ceph-mon[37701]: pgmap v1695: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:10.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:10 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:10.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:10 smithi167 ceph-mon[29496]: pgmap v1695: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:10.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:10.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:10 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:11 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:36:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:11.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:11 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:36:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:11.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:11.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:11 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:11.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:36:11.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:11.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:12 smithi179 ceph-mon[33312]: pgmap v1696: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:12 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:36:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:12 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:36:12.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:12 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:12 smithi167 ceph-mon[37701]: pgmap v1696: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:12 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:36:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:12 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:36:12.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:12 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:12.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:12 smithi167 ceph-mon[29496]: pgmap v1696: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:12.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:12 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:36:12.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:12 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:36:12.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:12 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:14.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:14 smithi179 ceph-mon[33312]: pgmap v1697: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:14.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:14 smithi167 ceph-mon[37701]: pgmap v1697: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:14.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:14 smithi167 ceph-mon[29496]: pgmap v1697: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:36:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:36:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:36:15.550 INFO:tasks.workunit.client.0.smithi167.stderr:+ pid=150743 2024-06-18T13:36:15.550 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool set-quota 2d48a5d0-5bf6-4059-9227-066b6e28e48a max_bytes 0 2024-06-18T13:36:15.551 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put two /etc/passwd 2024-06-18T13:36:15.644 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.642+0000 7f3dec772640 1 -- 172.21.15.167:0/2639055674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de4100f20 msgr2=0x7f3de410d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:15.644 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.642+0000 7f3dec772640 1 --2- 172.21.15.167:0/2639055674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de4100f20 0x7f3de410d040 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009ca0 tx=0x7f3ddc02c0f0 comp rx=0 tx=0).stop 2024-06-18T13:36:15.644 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.642+0000 7f3dec772640 1 -- 172.21.15.167:0/2639055674 shutdown_connections 2024-06-18T13:36:15.645 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.642+0000 7f3dec772640 1 --2- 172.21.15.167:0/2639055674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de4100f20 0x7f3de410d040 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:15.645 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.642+0000 7f3dec772640 1 --2- 172.21.15.167:0/2639055674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3de4100580 0x7f3de41009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:15.645 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.642+0000 7f3dec772640 1 --2- 172.21.15.167:0/2639055674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3de41051b0 0x7f3de4105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:15.645 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.642+0000 7f3dec772640 1 -- 172.21.15.167:0/2639055674 >> 172.21.15.167:0/2639055674 conn(0x7f3de4075f30 msgr2=0x7f3de4076340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:36:15.645 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.642+0000 7f3dec772640 1 -- 172.21.15.167:0/2639055674 shutdown_connections 2024-06-18T13:36:15.645 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.642+0000 7f3dec772640 1 -- 172.21.15.167:0/2639055674 wait complete. 2024-06-18T13:36:15.645 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.642+0000 7f3dec772640 1 Processor -- start 2024-06-18T13:36:15.645 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.642+0000 7f3dec772640 1 -- start start 2024-06-18T13:36:15.645 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dec772640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3de4100580 0x7f3de4197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:15.645 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dec772640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3de4100f20 0x7f3de41975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:15.645 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dec772640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de41051b0 0x7f3de419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:15.646 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dec772640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3de410f9f0 con 0x7f3de41051b0 2024-06-18T13:36:15.646 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dec772640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3de410f870 con 0x7f3de4100f20 2024-06-18T13:36:15.646 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dec772640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3de410fb70 con 0x7f3de4100580 2024-06-18T13:36:15.646 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dea50e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3de4100580 0x7f3de4197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:15.646 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dead0f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de41051b0 0x7f3de419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:15.646 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dea50e640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3de4100580 0x7f3de4197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:56018/0 (socket says 172.21.15.167:56018) 2024-06-18T13:36:15.646 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dea50e640 1 -- 172.21.15.167:0/617073738 learned_addr learned my addr 172.21.15.167:0/617073738 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:36:15.646 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3de9d0d640 1 --2- 172.21.15.167:0/617073738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3de4100f20 0x7f3de41975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:15.646 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dea50e640 1 -- 172.21.15.167:0/617073738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3de4100f20 msgr2=0x7f3de41975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:15.646 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dea50e640 1 --2- 172.21.15.167:0/617073738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3de4100f20 0x7f3de41975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:15.646 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dea50e640 1 -- 172.21.15.167:0/617073738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de41051b0 msgr2=0x7f3de419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:15.647 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dea50e640 1 --2- 172.21.15.167:0/617073738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de41051b0 0x7f3de419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:15.647 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dea50e640 1 -- 172.21.15.167:0/617073738 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de419c050 con 0x7f3de4100580 2024-06-18T13:36:15.647 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dead0f640 1 --2- 172.21.15.167:0/617073738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de41051b0 0x7f3de419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:36:15.647 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dea50e640 1 --2- 172.21.15.167:0/617073738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3de4100580 0x7f3de4197060 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400d950 tx=0x7f3dd400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:36:15.647 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3ddb7fe640 1 -- 172.21.15.167:0/617073738 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4017070 con 0x7f3de4100580 2024-06-18T13:36:15.647 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.643+0000 7f3dec772640 1 -- 172.21.15.167:0/617073738 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de419c340 con 0x7f3de4100580 2024-06-18T13:36:15.647 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.644+0000 7f3ddb7fe640 1 -- 172.21.15.167:0/617073738 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f3dd4009c30 con 0x7f3de4100580 2024-06-18T13:36:15.647 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.644+0000 7f3ddb7fe640 1 -- 172.21.15.167:0/617073738 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400f620 con 0x7f3de4100580 2024-06-18T13:36:15.647 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.644+0000 7f3dec772640 1 -- 172.21.15.167:0/617073738 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de41a3bd0 con 0x7f3de4100580 2024-06-18T13:36:15.647 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.644+0000 7f3dec772640 1 -- 172.21.15.167:0/617073738 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de41018b0 con 0x7f3de4100580 2024-06-18T13:36:15.648 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.645+0000 7f3ddb7fe640 1 -- 172.21.15.167:0/617073738 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f3dd4010460 con 0x7f3de4100580 2024-06-18T13:36:15.648 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.646+0000 7f3ddb7fe640 1 --2- 172.21.15.167:0/617073738 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f3dd006e050 0x7f3dd0070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:15.648 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.646+0000 7f3de9d0d640 1 --2- 172.21.15.167:0/617073738 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f3dd006e050 0x7f3dd0070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:15.648 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.646+0000 7f3ddb7fe640 1 -- 172.21.15.167:0/617073738 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(772..772 src has 257..772) ==== 9092+0+0 (secure 0 0 0) 0x7f3dd408e8d0 con 0x7f3de4100580 2024-06-18T13:36:15.648 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.646+0000 7f3ddb7fe640 1 -- 172.21.15.167:0/617073738 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=773}) -- 0x7f3dd0079bb0 con 0x7f3de4100580 2024-06-18T13:36:15.648 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.646+0000 7f3de9d0d640 1 --2- 172.21.15.167:0/617073738 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f3dd006e050 0x7f3dd0070510 secure :-1 s=READY pgs=3505 cs=0 l=1 rev1=1 crypto rx=0x7f3de00095f0 tx=0x7f3de0005f90 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:36:15.654 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.654+0000 7f3ddb7fe640 1 -- 172.21.15.167:0/617073738 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f3dd405f2a0 con 0x7f3de4100580 2024-06-18T13:36:15.835 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:15.835+0000 7f3dec772640 1 -- 172.21.15.167:0/617073738 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"} v 0) -- 0x7f3de41983b0 con 0x7f3de4100580 2024-06-18T13:36:16.613 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:16.613+0000 7f3ddb7fe640 1 -- 172.21.15.167:0/617073738 <== mon.2 v2:172.21.15.167:3301/0 7 ==== osd_map(773..773 src has 257..773) ==== 628+0+0 (secure 0 0 0) 0x7f3dd4059040 con 0x7f3de4100580 2024-06-18T13:36:16.613 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:16.613+0000 7f3ddb7fe640 1 -- 172.21.15.167:0/617073738 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=774}) -- 0x7f3dd007a6c0 con 0x7f3de4100580 2024-06-18T13:36:16.616 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:16.616+0000 7f3ddb7fe640 1 -- 172.21.15.167:0/617073738 <== mon.2 v2:172.21.15.167:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a v773) ==== 217+0+0 (secure 0 0 0) 0x7f3dd4063ad0 con 0x7f3de4100580 2024-06-18T13:36:16.727 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:16.727+0000 7f3dec772640 1 -- 172.21.15.167:0/617073738 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"} v 0) -- 0x7f3de41a3ec0 con 0x7f3de4100580 2024-06-18T13:36:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:16 smithi179 ceph-mon[33312]: pgmap v1698: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:16 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/617073738' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:36:16.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:16 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:36:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:16 smithi167 ceph-mon[37701]: pgmap v1698: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:16 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/617073738' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:36:16.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:16 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:36:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:16 smithi167 ceph-mon[29496]: pgmap v1698: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:16 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/617073738' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:36:16.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:16 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:36:17.615 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.615+0000 7f3ddb7fe640 1 -- 172.21.15.167:0/617073738 <== mon.2 v2:172.21.15.167:3301/0 9 ==== osd_map(774..774 src has 257..774) ==== 628+0+0 (secure 0 0 0) 0x7f3dd408c150 con 0x7f3de4100580 2024-06-18T13:36:17.616 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.615+0000 7f3ddb7fe640 1 -- 172.21.15.167:0/617073738 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=775}) -- 0x7f3dd007acc0 con 0x7f3de4100580 2024-06-18T13:36:17.622 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.621+0000 7f3ddb7fe640 1 -- 172.21.15.167:0/617073738 <== mon.2 v2:172.21.15.167:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a v774) ==== 217+0+0 (secure 0 0 0) 0x7f3dd40592a0 con 0x7f3de4100580 2024-06-18T13:36:17.622 INFO:tasks.workunit.client.0.smithi167.stderr:set-quota max_bytes = 0 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a 2024-06-18T13:36:17.626 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.625+0000 7f3dec772640 1 -- 172.21.15.167:0/617073738 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f3dd006e050 msgr2=0x7f3dd0070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:17.626 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.625+0000 7f3dec772640 1 --2- 172.21.15.167:0/617073738 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f3dd006e050 0x7f3dd0070510 secure :-1 s=READY pgs=3505 cs=0 l=1 rev1=1 crypto rx=0x7f3de00095f0 tx=0x7f3de0005f90 comp rx=0 tx=0).stop 2024-06-18T13:36:17.626 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.625+0000 7f3dec772640 1 -- 172.21.15.167:0/617073738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3de4100580 msgr2=0x7f3de4197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:17.626 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.625+0000 7f3dec772640 1 --2- 172.21.15.167:0/617073738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3de4100580 0x7f3de4197060 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400d950 tx=0x7f3dd400de10 comp rx=0 tx=0).stop 2024-06-18T13:36:17.626 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.625+0000 7f3dec772640 1 -- 172.21.15.167:0/617073738 shutdown_connections 2024-06-18T13:36:17.626 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.625+0000 7f3dec772640 1 --2- 172.21.15.167:0/617073738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de41051b0 0x7f3de419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:17.626 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.625+0000 7f3dec772640 1 --2- 172.21.15.167:0/617073738 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f3dd006e050 0x7f3dd0070510 unknown :-1 s=CLOSED pgs=3505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:17.626 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.625+0000 7f3dec772640 1 --2- 172.21.15.167:0/617073738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3de4100f20 0x7f3de41975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:17.626 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.625+0000 7f3dec772640 1 --2- 172.21.15.167:0/617073738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3de4100580 0x7f3de4197060 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:17.626 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.625+0000 7f3dec772640 1 -- 172.21.15.167:0/617073738 >> 172.21.15.167:0/617073738 conn(0x7f3de4075f30 msgr2=0x7f3de4068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:36:17.627 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.625+0000 7f3dec772640 1 -- 172.21.15.167:0/617073738 shutdown_connections 2024-06-18T13:36:17.627 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.625+0000 7f3dec772640 1 -- 172.21.15.167:0/617073738 wait complete. 2024-06-18T13:36:17.636 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool set-quota 2d48a5d0-5bf6-4059-9227-066b6e28e48a max_objects 0 2024-06-18T13:36:17.728 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.725+0000 7ff496037640 1 -- 172.21.15.167:0/1078038686 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff490100400 msgr2=0x7ff4901007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:17.728 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.725+0000 7ff496037640 1 --2- 172.21.15.167:0/1078038686 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff490100400 0x7ff4901007e0 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7ff480009860 tx=0x7ff48002bb70 comp rx=0 tx=0).stop 2024-06-18T13:36:17.728 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.725+0000 7ff496037640 1 -- 172.21.15.167:0/1078038686 shutdown_connections 2024-06-18T13:36:17.728 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.725+0000 7ff496037640 1 --2- 172.21.15.167:0/1078038686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff490105550 0x7ff49010d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:17.728 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.725+0000 7ff496037640 1 --2- 172.21.15.167:0/1078038686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff490100db0 0x7ff490104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:17.728 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.725+0000 7ff496037640 1 --2- 172.21.15.167:0/1078038686 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff490100400 0x7ff4901007e0 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:17.728 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.725+0000 7ff496037640 1 -- 172.21.15.167:0/1078038686 >> 172.21.15.167:0/1078038686 conn(0x7ff4900fbe70 msgr2=0x7ff4900fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:36:17.728 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.725+0000 7ff496037640 1 -- 172.21.15.167:0/1078038686 shutdown_connections 2024-06-18T13:36:17.728 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.725+0000 7ff496037640 1 -- 172.21.15.167:0/1078038686 wait complete. 2024-06-18T13:36:17.728 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.725+0000 7ff496037640 1 Processor -- start 2024-06-18T13:36:17.728 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff496037640 1 -- start start 2024-06-18T13:36:17.729 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff496037640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff490100400 0x7ff490197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:17.729 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff496037640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff490105550 0x7ff4901976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:17.729 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff496037640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff49019ba40 0x7ff49019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:17.729 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff496037640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff49010fa10 con 0x7ff49019ba40 2024-06-18T13:36:17.729 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff496037640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff49010f890 con 0x7ff490100400 2024-06-18T13:36:17.729 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff496037640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff49010fb90 con 0x7ff490105550 2024-06-18T13:36:17.729 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff48ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff49019ba40 0x7ff49019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:17.729 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff48effd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff490105550 0x7ff4901976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:17.729 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff48ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff49019ba40 0x7ff49019bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46336/0 (socket says 172.21.15.167:46336) 2024-06-18T13:36:17.729 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff48effd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff490105550 0x7ff4901976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:56032/0 (socket says 172.21.15.167:56032) 2024-06-18T13:36:17.729 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff48ffff640 1 -- 172.21.15.167:0/2876898854 learned_addr learned my addr 172.21.15.167:0/2876898854 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:36:17.729 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff48f7fe640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff490100400 0x7ff490197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:17.730 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff48effd640 1 -- 172.21.15.167:0/2876898854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff490100400 msgr2=0x7ff490197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:17.730 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff48effd640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff490100400 0x7ff490197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:17.730 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff48effd640 1 -- 172.21.15.167:0/2876898854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff49019ba40 msgr2=0x7ff49019bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:17.730 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff48effd640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff49019ba40 0x7ff49019bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:17.730 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff48effd640 1 -- 172.21.15.167:0/2876898854 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49019c530 con 0x7ff490105550 2024-06-18T13:36:17.730 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff48f7fe640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff490100400 0x7ff490197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T13:36:17.730 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff48ffff640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff49019ba40 0x7ff49019bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:36:17.730 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.726+0000 7ff48effd640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff490105550 0x7ff4901976c0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7ff48400c940 tx=0x7ff48400ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:36:17.730 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.727+0000 7ff48cff9640 1 -- 172.21.15.167:0/2876898854 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484007bb0 con 0x7ff490105550 2024-06-18T13:36:17.730 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.727+0000 7ff48cff9640 1 -- 172.21.15.167:0/2876898854 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7ff484007d50 con 0x7ff490105550 2024-06-18T13:36:17.730 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.727+0000 7ff496037640 1 -- 172.21.15.167:0/2876898854 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49019c820 con 0x7ff490105550 2024-06-18T13:36:17.730 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.727+0000 7ff48cff9640 1 -- 172.21.15.167:0/2876898854 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484004cc0 con 0x7ff490105550 2024-06-18T13:36:17.730 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.727+0000 7ff496037640 1 -- 172.21.15.167:0/2876898854 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4901a4010 con 0x7ff490105550 2024-06-18T13:36:17.731 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.728+0000 7ff496037640 1 -- 172.21.15.167:0/2876898854 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4901007e0 con 0x7ff490105550 2024-06-18T13:36:17.731 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.728+0000 7ff48cff9640 1 -- 172.21.15.167:0/2876898854 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7ff484004ea0 con 0x7ff490105550 2024-06-18T13:36:17.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.729+0000 7ff48cff9640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7ff47806e050 0x7ff478070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:17.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.729+0000 7ff48f7fe640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7ff47806e050 0x7ff478070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:17.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.729+0000 7ff48cff9640 1 -- 172.21.15.167:0/2876898854 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(774..774 src has 257..774) ==== 9092+0+0 (secure 0 0 0) 0x7ff48408e910 con 0x7ff490105550 2024-06-18T13:36:17.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.729+0000 7ff48cff9640 1 -- 172.21.15.167:0/2876898854 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=775}) -- 0x7ff478079bd0 con 0x7ff490105550 2024-06-18T13:36:17.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.729+0000 7ff48f7fe640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7ff47806e050 0x7ff478070510 secure :-1 s=READY pgs=3506 cs=0 l=1 rev1=1 crypto rx=0x7ff48002c180 tx=0x7ff4800058b0 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:36:17.733 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.733+0000 7ff48cff9640 1 -- 172.21.15.167:0/2876898854 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7ff484014030 con 0x7ff490105550 2024-06-18T13:36:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]': finished 2024-06-18T13:36:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:17 smithi179 ceph-mon[33312]: osdmap e773: 8 total, 8 up, 8 in 2024-06-18T13:36:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:17 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/617073738' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:36:17.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:17 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:36:17.910 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:17.910+0000 7ff496037640 1 -- 172.21.15.167:0/2876898854 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"} v 0) -- 0x7ff490100db0 con 0x7ff490105550 2024-06-18T13:36:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]': finished 2024-06-18T13:36:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:17 smithi167 ceph-mon[37701]: osdmap e773: 8 total, 8 up, 8 in 2024-06-18T13:36:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:17 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/617073738' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:36:17.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:17 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:36:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]': finished 2024-06-18T13:36:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:17 smithi167 ceph-mon[29496]: osdmap e773: 8 total, 8 up, 8 in 2024-06-18T13:36:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:17 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/617073738' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:36:17.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:17 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:36:18.453 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:18.452+0000 7ff48cff9640 1 -- 172.21.15.167:0/2876898854 <== mon.2 v2:172.21.15.167:3301/0 7 ==== osd_map(775..775 src has 257..775) ==== 628+0+0 (secure 0 0 0) 0x7ff484059100 con 0x7ff490105550 2024-06-18T13:36:18.453 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:18.452+0000 7ff48cff9640 1 -- 172.21.15.167:0/2876898854 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=776}) -- 0x7ff47807a6e0 con 0x7ff490105550 2024-06-18T13:36:18.457 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:18.457+0000 7ff48cff9640 1 -- 172.21.15.167:0/2876898854 <== mon.2 v2:172.21.15.167:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a v775) ==== 221+0+0 (secure 0 0 0) 0x7ff48405f360 con 0x7ff490105550 2024-06-18T13:36:18.570 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:18.570+0000 7ff496037640 1 -- 172.21.15.167:0/2876898854 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"} v 0) -- 0x7ff4901a4240 con 0x7ff490105550 2024-06-18T13:36:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:18 smithi179 ceph-mon[33312]: pgmap v1700: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:36:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]': finished 2024-06-18T13:36:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:18 smithi179 ceph-mon[33312]: osdmap e774: 8 total, 8 up, 8 in 2024-06-18T13:36:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2876898854' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:36:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:36:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:18 smithi179 ceph-mon[33312]: pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' no longer out of quota; removing NO_QUOTA flag 2024-06-18T13:36:18.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:18 smithi179 ceph-mon[33312]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-18T13:36:18.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]': finished 2024-06-18T13:36:18.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:18 smithi179 ceph-mon[33312]: osdmap e775: 8 total, 8 up, 8 in 2024-06-18T13:36:18.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:18 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2876898854' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:36:18.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:18 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:36:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[37701]: pgmap v1700: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:36:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]': finished 2024-06-18T13:36:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[37701]: osdmap e774: 8 total, 8 up, 8 in 2024-06-18T13:36:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2876898854' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:36:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:36:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[37701]: pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' no longer out of quota; removing NO_QUOTA flag 2024-06-18T13:36:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[37701]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-18T13:36:18.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]': finished 2024-06-18T13:36:18.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[37701]: osdmap e775: 8 total, 8 up, 8 in 2024-06-18T13:36:18.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2876898854' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:36:18.978 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:36:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[29496]: pgmap v1700: 176 pgs: 176 active+clean; 724 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:36:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]': finished 2024-06-18T13:36:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[29496]: osdmap e774: 8 total, 8 up, 8 in 2024-06-18T13:36:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2876898854' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:36:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:36:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[29496]: pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' no longer out of quota; removing NO_QUOTA flag 2024-06-18T13:36:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[29496]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-18T13:36:18.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]': finished 2024-06-18T13:36:18.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[29496]: osdmap e775: 8 total, 8 up, 8 in 2024-06-18T13:36:18.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2876898854' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:36:18.979 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:18 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:36:19.455 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.455+0000 7ff48cff9640 1 -- 172.21.15.167:0/2876898854 <== mon.2 v2:172.21.15.167:3301/0 9 ==== osd_map(776..776 src has 257..776) ==== 628+0+0 (secure 0 0 0) 0x7ff48408c190 con 0x7ff490105550 2024-06-18T13:36:19.460 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.459+0000 7ff48cff9640 1 -- 172.21.15.167:0/2876898854 <== mon.2 v2:172.21.15.167:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a v776) ==== 221+0+0 (secure 0 0 0) 0x7ff484063b90 con 0x7ff490105550 2024-06-18T13:36:19.460 INFO:tasks.workunit.client.0.smithi167.stderr:set-quota max_objects = 0 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a 2024-06-18T13:36:19.463 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.463+0000 7ff496037640 1 -- 172.21.15.167:0/2876898854 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7ff47806e050 msgr2=0x7ff478070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:19.463 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.463+0000 7ff496037640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7ff47806e050 0x7ff478070510 secure :-1 s=READY pgs=3506 cs=0 l=1 rev1=1 crypto rx=0x7ff48002c180 tx=0x7ff4800058b0 comp rx=0 tx=0).stop 2024-06-18T13:36:19.464 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.463+0000 7ff496037640 1 -- 172.21.15.167:0/2876898854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff490105550 msgr2=0x7ff4901976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:19.464 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.463+0000 7ff496037640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff490105550 0x7ff4901976c0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7ff48400c940 tx=0x7ff48400ce00 comp rx=0 tx=0).stop 2024-06-18T13:36:19.464 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.463+0000 7ff496037640 1 -- 172.21.15.167:0/2876898854 shutdown_connections 2024-06-18T13:36:19.464 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.463+0000 7ff496037640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff49019ba40 0x7ff49019bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:19.464 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.463+0000 7ff496037640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff490105550 0x7ff4901976c0 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:19.464 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.463+0000 7ff496037640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7ff47806e050 0x7ff478070510 unknown :-1 s=CLOSED pgs=3506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:19.464 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.463+0000 7ff496037640 1 --2- 172.21.15.167:0/2876898854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff490100400 0x7ff490197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:19.464 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.463+0000 7ff496037640 1 -- 172.21.15.167:0/2876898854 >> 172.21.15.167:0/2876898854 conn(0x7ff4900fbe70 msgr2=0x7ff4901042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:36:19.464 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.463+0000 7ff496037640 1 -- 172.21.15.167:0/2876898854 shutdown_connections 2024-06-18T13:36:19.464 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.463+0000 7ff496037640 1 -- 172.21.15.167:0/2876898854 wait complete. 2024-06-18T13:36:19.475 INFO:tasks.workunit.client.0.smithi167.stderr:+ wait 150743 2024-06-18T13:36:19.475 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 0 -ne 0 ']' 2024-06-18T13:36:19.475 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2024-06-18T13:36:19.475 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put three /etc/passwd 2024-06-18T13:36:19.513 INFO:tasks.workunit.client.0.smithi167.stderr:++ uuidgen 2024-06-18T13:36:19.514 INFO:tasks.workunit.client.0.smithi167.stderr:+ pp=281aa14d-36f8-4e55-8e76-736d78c61052 2024-06-18T13:36:19.514 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool create 281aa14d-36f8-4e55-8e76-736d78c61052 12 2024-06-18T13:36:19.600 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.598+0000 7fad2064b640 1 -- 172.21.15.167:0/2370171528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad181051b0 msgr2=0x7fad18105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:19.600 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.598+0000 7fad2064b640 1 --2- 172.21.15.167:0/2370171528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad181051b0 0x7fad18105590 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009900 tx=0x7fad0c02bc30 comp rx=0 tx=0).stop 2024-06-18T13:36:19.600 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.598+0000 7fad2064b640 1 -- 172.21.15.167:0/2370171528 shutdown_connections 2024-06-18T13:36:19.601 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.598+0000 7fad2064b640 1 --2- 172.21.15.167:0/2370171528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad18100f20 0x7fad1810d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:19.601 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.598+0000 7fad2064b640 1 --2- 172.21.15.167:0/2370171528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad18100580 0x7fad181009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:19.601 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.598+0000 7fad2064b640 1 --2- 172.21.15.167:0/2370171528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad181051b0 0x7fad18105590 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:19.601 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.598+0000 7fad2064b640 1 -- 172.21.15.167:0/2370171528 >> 172.21.15.167:0/2370171528 conn(0x7fad18075f30 msgr2=0x7fad18076340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:36:19.601 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.598+0000 7fad2064b640 1 -- 172.21.15.167:0/2370171528 shutdown_connections 2024-06-18T13:36:19.601 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.598+0000 7fad2064b640 1 -- 172.21.15.167:0/2370171528 wait complete. 2024-06-18T13:36:19.601 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.598+0000 7fad2064b640 1 Processor -- start 2024-06-18T13:36:19.601 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad2064b640 1 -- start start 2024-06-18T13:36:19.601 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad2064b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad18100580 0x7fad18194e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:19.601 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad2064b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad18100f20 0x7fad181953b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:19.601 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad2064b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad181051b0 0x7fad18199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:19.602 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad2064b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fad1810e940 con 0x7fad18100580 2024-06-18T13:36:19.602 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad2064b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fad1810e7c0 con 0x7fad18100f20 2024-06-18T13:36:19.602 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad2064b640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fad1810eac0 con 0x7fad181051b0 2024-06-18T13:36:19.602 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad1e3e7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad18100580 0x7fad18194e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:19.602 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad1e3e7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad18100580 0x7fad18194e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46368/0 (socket says 172.21.15.167:46368) 2024-06-18T13:36:19.602 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad1e3e7640 1 -- 172.21.15.167:0/1584530233 learned_addr learned my addr 172.21.15.167:0/1584530233 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:36:19.602 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad1ebe8640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad181051b0 0x7fad18199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:19.602 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad1dbe6640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad18100f20 0x7fad181953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:19.602 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad1e3e7640 1 -- 172.21.15.167:0/1584530233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad181051b0 msgr2=0x7fad18199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:19.602 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad1e3e7640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad181051b0 0x7fad18199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:19.603 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad1e3e7640 1 -- 172.21.15.167:0/1584530233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad18100f20 msgr2=0x7fad181953b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:19.603 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad1e3e7640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad18100f20 0x7fad181953b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:19.603 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad1e3e7640 1 -- 172.21.15.167:0/1584530233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad18199e60 con 0x7fad18100580 2024-06-18T13:36:19.603 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad1dbe6640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad18100f20 0x7fad181953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-18T13:36:19.603 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad1e3e7640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad18100580 0x7fad18194e70 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fad0c005940 tx=0x7fad0c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:36:19.603 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad0b7fe640 1 -- 172.21.15.167:0/1584530233 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c043020 con 0x7fad18100580 2024-06-18T13:36:19.603 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad2064b640 1 -- 172.21.15.167:0/1584530233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1819a0f0 con 0x7fad18100580 2024-06-18T13:36:19.603 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad0b7fe640 1 -- 172.21.15.167:0/1584530233 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fad0c005480 con 0x7fad18100580 2024-06-18T13:36:19.603 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.599+0000 7fad0b7fe640 1 -- 172.21.15.167:0/1584530233 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c003890 con 0x7fad18100580 2024-06-18T13:36:19.603 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.600+0000 7fad2064b640 1 -- 172.21.15.167:0/1584530233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad181a19e0 con 0x7fad18100580 2024-06-18T13:36:19.604 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.601+0000 7fad2064b640 1 -- 172.21.15.167:0/1584530233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad181018b0 con 0x7fad18100580 2024-06-18T13:36:19.605 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.601+0000 7fad0b7fe640 1 -- 172.21.15.167:0/1584530233 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7fad0c046020 con 0x7fad18100580 2024-06-18T13:36:19.605 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.601+0000 7fad0b7fe640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fad0406e050 0x7fad04070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:19.605 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.601+0000 7fad0b7fe640 1 -- 172.21.15.167:0/1584530233 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(776..776 src has 257..776) ==== 9092+0+0 (secure 0 0 0) 0x7fad0c0b0d30 con 0x7fad18100580 2024-06-18T13:36:19.605 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.601+0000 7fad1dbe6640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fad0406e050 0x7fad04070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:19.605 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.602+0000 7fad1dbe6640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fad0406e050 0x7fad04070510 secure :-1 s=READY pgs=3508 cs=0 l=1 rev1=1 crypto rx=0x7fad18076510 tx=0x7fad1400f040 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:36:19.606 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.606+0000 7fad0b7fe640 1 -- 172.21.15.167:0/1584530233 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fad0c04d020 con 0x7fad18100580 2024-06-18T13:36:19.783 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:19.784+0000 7fad2064b640 1 -- 172.21.15.167:0/1584530233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pg_num": 12} v 0) -- 0x7fad181961c0 con 0x7fad18100580 2024-06-18T13:36:20.619 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.619+0000 7fad0b7fe640 1 -- 172.21.15.167:0/1584530233 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pg_num": 12}]=0 pool '281aa14d-36f8-4e55-8e76-736d78c61052' created v777) ==== 176+0+0 (secure 0 0 0) 0x7fad0c07db70 con 0x7fad18100580 2024-06-18T13:36:20.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:36:20 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:36:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:20 smithi179 ceph-mon[33312]: pgmap v1703: 176 pgs: 176 active+clean; 724 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:36:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:20 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]': finished 2024-06-18T13:36:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:20 smithi179 ceph-mon[33312]: osdmap e776: 8 total, 8 up, 8 in 2024-06-18T13:36:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:20 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1584530233' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pg_num": 12}]: dispatch 2024-06-18T13:36:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:20 smithi167 ceph-mon[37701]: pgmap v1703: 176 pgs: 176 active+clean; 724 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:36:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:20 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]': finished 2024-06-18T13:36:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:20 smithi167 ceph-mon[37701]: osdmap e776: 8 total, 8 up, 8 in 2024-06-18T13:36:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:20 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1584530233' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pg_num": 12}]: dispatch 2024-06-18T13:36:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:20 smithi167 ceph-mon[29496]: pgmap v1703: 176 pgs: 176 active+clean; 724 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-18T13:36:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:20 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]': finished 2024-06-18T13:36:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:20 smithi167 ceph-mon[29496]: osdmap e776: 8 total, 8 up, 8 in 2024-06-18T13:36:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:20 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1584530233' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pg_num": 12}]: dispatch 2024-06-18T13:36:20.729 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.729+0000 7fad2064b640 1 -- 172.21.15.167:0/1584530233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pg_num": 12} v 0) -- 0x7fad18108700 con 0x7fad18100580 2024-06-18T13:36:20.729 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.729+0000 7fad0b7fe640 1 -- 172.21.15.167:0/1584530233 <== mon.0 v2:172.21.15.167:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pg_num": 12}]=0 pool '281aa14d-36f8-4e55-8e76-736d78c61052' already exists v777) ==== 183+0+0 (secure 0 0 0) 0x7fad0c081820 con 0x7fad18100580 2024-06-18T13:36:20.730 INFO:tasks.workunit.client.0.smithi167.stderr:pool '281aa14d-36f8-4e55-8e76-736d78c61052' already exists 2024-06-18T13:36:20.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.731+0000 7fad2064b640 1 -- 172.21.15.167:0/1584530233 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fad0406e050 msgr2=0x7fad04070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:20.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.731+0000 7fad2064b640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fad0406e050 0x7fad04070510 secure :-1 s=READY pgs=3508 cs=0 l=1 rev1=1 crypto rx=0x7fad18076510 tx=0x7fad1400f040 comp rx=0 tx=0).stop 2024-06-18T13:36:20.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.731+0000 7fad2064b640 1 -- 172.21.15.167:0/1584530233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad18100580 msgr2=0x7fad18194e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:20.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.731+0000 7fad2064b640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad18100580 0x7fad18194e70 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fad0c005940 tx=0x7fad0c004ab0 comp rx=0 tx=0).stop 2024-06-18T13:36:20.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.731+0000 7fad2064b640 1 -- 172.21.15.167:0/1584530233 shutdown_connections 2024-06-18T13:36:20.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.731+0000 7fad2064b640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad181051b0 0x7fad18199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:20.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.731+0000 7fad2064b640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fad0406e050 0x7fad04070510 unknown :-1 s=CLOSED pgs=3508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:20.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.731+0000 7fad2064b640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad18100f20 0x7fad181953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:20.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.731+0000 7fad2064b640 1 --2- 172.21.15.167:0/1584530233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad18100580 0x7fad18194e70 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:20.732 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.731+0000 7fad2064b640 1 -- 172.21.15.167:0/1584530233 >> 172.21.15.167:0/1584530233 conn(0x7fad18075f30 msgr2=0x7fad180fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:36:20.733 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.732+0000 7fad2064b640 1 -- 172.21.15.167:0/1584530233 shutdown_connections 2024-06-18T13:36:20.733 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.732+0000 7fad2064b640 1 -- 172.21.15.167:0/1584530233 wait complete. 2024-06-18T13:36:20.740 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool application enable 281aa14d-36f8-4e55-8e76-736d78c61052 rados 2024-06-18T13:36:20.826 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.823+0000 7f9a55e64640 1 -- 172.21.15.167:0/2300693357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a50102de0 msgr2=0x7f9a5010b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:20.826 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.823+0000 7f9a55e64640 1 --2- 172.21.15.167:0/2300693357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a50102de0 0x7f9a5010b2c0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009900 tx=0x7f9a4402bcd0 comp rx=0 tx=0).stop 2024-06-18T13:36:20.826 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.823+0000 7f9a55e64640 1 -- 172.21.15.167:0/2300693357 shutdown_connections 2024-06-18T13:36:20.826 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.823+0000 7f9a55e64640 1 --2- 172.21.15.167:0/2300693357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a5010b890 0x7f9a5010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:20.826 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.823+0000 7f9a55e64640 1 --2- 172.21.15.167:0/2300693357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a50102de0 0x7f9a5010b2c0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:20.826 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.823+0000 7f9a55e64640 1 --2- 172.21.15.167:0/2300693357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a501024c0 0x7f9a501028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:20.826 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.823+0000 7f9a55e64640 1 -- 172.21.15.167:0/2300693357 >> 172.21.15.167:0/2300693357 conn(0x7f9a500fbe70 msgr2=0x7f9a500fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:36:20.826 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.823+0000 7f9a55e64640 1 -- 172.21.15.167:0/2300693357 shutdown_connections 2024-06-18T13:36:20.826 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.823+0000 7f9a55e64640 1 -- 172.21.15.167:0/2300693357 wait complete. 2024-06-18T13:36:20.827 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.824+0000 7f9a55e64640 1 Processor -- start 2024-06-18T13:36:20.827 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.824+0000 7f9a55e64640 1 -- start start 2024-06-18T13:36:20.827 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.824+0000 7f9a55e64640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a501024c0 0x7f9a50100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:20.827 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.824+0000 7f9a55e64640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a50102de0 0x7f9a50100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:20.827 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.824+0000 7f9a55e64640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a5010b890 0x7f9a50100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:20.827 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.824+0000 7f9a55e64640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9a50110ae0 con 0x7f9a5010b890 2024-06-18T13:36:20.827 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.824+0000 7f9a55e64640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9a50110960 con 0x7f9a501024c0 2024-06-18T13:36:20.827 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.824+0000 7f9a55e64640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9a50110c60 con 0x7f9a50102de0 2024-06-18T13:36:20.827 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a5010b890 0x7f9a50100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:20.827 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4effd640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a50102de0 0x7f9a50100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:20.827 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a5010b890 0x7f9a50100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46394/0 (socket says 172.21.15.167:46394) 2024-06-18T13:36:20.827 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4ffff640 1 -- 172.21.15.167:0/2602855275 learned_addr learned my addr 172.21.15.167:0/2602855275 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:36:20.828 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4f7fe640 1 --2- 172.21.15.167:0/2602855275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a501024c0 0x7f9a50100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:20.828 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4ffff640 1 -- 172.21.15.167:0/2602855275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a50102de0 msgr2=0x7f9a50100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:20.828 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4ffff640 1 --2- 172.21.15.167:0/2602855275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a50102de0 0x7f9a50100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:20.828 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4ffff640 1 -- 172.21.15.167:0/2602855275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a501024c0 msgr2=0x7f9a50100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:20.828 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4ffff640 1 --2- 172.21.15.167:0/2602855275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a501024c0 0x7f9a50100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:20.828 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4ffff640 1 -- 172.21.15.167:0/2602855275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a501a5670 con 0x7f9a5010b890 2024-06-18T13:36:20.828 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4effd640 1 --2- 172.21.15.167:0/2602855275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a50102de0 0x7f9a50100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:36:20.828 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4ffff640 1 --2- 172.21.15.167:0/2602855275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a5010b890 0x7f9a50100b90 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800eaa0 tx=0x7f9a4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:36:20.828 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4cff9640 1 -- 172.21.15.167:0/2602855275 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4800cdb0 con 0x7f9a5010b890 2024-06-18T13:36:20.828 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a55e64640 1 -- 172.21.15.167:0/2602855275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a501a5960 con 0x7f9a5010b890 2024-06-18T13:36:20.828 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4cff9640 1 -- 172.21.15.167:0/2602855275 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f9a48005020 con 0x7f9a5010b890 2024-06-18T13:36:20.828 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a4cff9640 1 -- 172.21.15.167:0/2602855275 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a480116d0 con 0x7f9a5010b890 2024-06-18T13:36:20.828 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.825+0000 7f9a55e64640 1 -- 172.21.15.167:0/2602855275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a501a5e80 con 0x7f9a5010b890 2024-06-18T13:36:20.829 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.826+0000 7f9a55e64640 1 -- 172.21.15.167:0/2602855275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a501028a0 con 0x7f9a5010b890 2024-06-18T13:36:20.829 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.827+0000 7f9a4cff9640 1 -- 172.21.15.167:0/2602855275 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f9a48004a40 con 0x7f9a5010b890 2024-06-18T13:36:20.829 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.827+0000 7f9a4cff9640 1 --2- 172.21.15.167:0/2602855275 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f9a3806e050 0x7f9a38070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:20.829 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.827+0000 7f9a4f7fe640 1 --2- 172.21.15.167:0/2602855275 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f9a3806e050 0x7f9a38070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:20.829 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.827+0000 7f9a4cff9640 1 -- 172.21.15.167:0/2602855275 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(777..777 src has 257..777) ==== 9467+0+0 (secure 0 0 0) 0x7f9a4808efa0 con 0x7f9a5010b890 2024-06-18T13:36:20.830 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.828+0000 7f9a4f7fe640 1 --2- 172.21.15.167:0/2602855275 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f9a3806e050 0x7f9a38070510 secure :-1 s=READY pgs=3509 cs=0 l=1 rev1=1 crypto rx=0x7f9a400095f0 tx=0x7f9a40005f90 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:36:20.832 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:20.832+0000 7f9a4cff9640 1 -- 172.21.15.167:0/2602855275 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f9a48016030 con 0x7f9a5010b890 2024-06-18T13:36:21.008 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:21.008+0000 7f9a55e64640 1 -- 172.21.15.167:0/2602855275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"} v 0) -- 0x7f9a501a6140 con 0x7f9a5010b890 2024-06-18T13:36:21.626 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:21.626+0000 7f9a4cff9640 1 -- 172.21.15.167:0/2602855275 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"}]=0 enabled application 'rados' on pool '281aa14d-36f8-4e55-8e76-736d78c61052' v778) ==== 213+0+0 (secure 0 0 0) 0x7f9a4805f800 con 0x7f9a5010b890 2024-06-18T13:36:21.745 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:21.744+0000 7f9a55e64640 1 -- 172.21.15.167:0/2602855275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"} v 0) -- 0x7f9a501a6310 con 0x7f9a5010b890 2024-06-18T13:36:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:21 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1584530233' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pg_num": 12}]': finished 2024-06-18T13:36:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:21 smithi179 ceph-mon[33312]: osdmap e777: 8 total, 8 up, 8 in 2024-06-18T13:36:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1584530233' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pg_num": 12}]: dispatch 2024-06-18T13:36:21.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:21 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2602855275' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"}]: dispatch 2024-06-18T13:36:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:21 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1584530233' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pg_num": 12}]': finished 2024-06-18T13:36:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:21 smithi167 ceph-mon[37701]: osdmap e777: 8 total, 8 up, 8 in 2024-06-18T13:36:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1584530233' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pg_num": 12}]: dispatch 2024-06-18T13:36:21.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:21 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2602855275' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"}]: dispatch 2024-06-18T13:36:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:21 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:21.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1584530233' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pg_num": 12}]': finished 2024-06-18T13:36:21.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:21 smithi167 ceph-mon[29496]: osdmap e777: 8 total, 8 up, 8 in 2024-06-18T13:36:21.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1584530233' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pg_num": 12}]: dispatch 2024-06-18T13:36:21.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:21 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2602855275' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"}]: dispatch 2024-06-18T13:36:22.636 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.636+0000 7f9a4cff9640 1 -- 172.21.15.167:0/2602855275 <== mon.0 v2:172.21.15.167:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"}]=0 enabled application 'rados' on pool '281aa14d-36f8-4e55-8e76-736d78c61052' v779) ==== 213+0+0 (secure 0 0 0) 0x7f9a48064030 con 0x7f9a5010b890 2024-06-18T13:36:22.636 INFO:tasks.workunit.client.0.smithi167.stderr:enabled application 'rados' on pool '281aa14d-36f8-4e55-8e76-736d78c61052' 2024-06-18T13:36:22.639 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.639+0000 7f9a55e64640 1 -- 172.21.15.167:0/2602855275 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f9a3806e050 msgr2=0x7f9a38070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:22.639 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.639+0000 7f9a55e64640 1 --2- 172.21.15.167:0/2602855275 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f9a3806e050 0x7f9a38070510 secure :-1 s=READY pgs=3509 cs=0 l=1 rev1=1 crypto rx=0x7f9a400095f0 tx=0x7f9a40005f90 comp rx=0 tx=0).stop 2024-06-18T13:36:22.640 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.639+0000 7f9a55e64640 1 -- 172.21.15.167:0/2602855275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a5010b890 msgr2=0x7f9a50100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:22.640 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.639+0000 7f9a55e64640 1 --2- 172.21.15.167:0/2602855275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a5010b890 0x7f9a50100b90 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800eaa0 tx=0x7f9a4800ef60 comp rx=0 tx=0).stop 2024-06-18T13:36:22.640 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.639+0000 7f9a55e64640 1 -- 172.21.15.167:0/2602855275 shutdown_connections 2024-06-18T13:36:22.640 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.639+0000 7f9a55e64640 1 --2- 172.21.15.167:0/2602855275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a5010b890 0x7f9a50100b90 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:22.640 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.639+0000 7f9a55e64640 1 --2- 172.21.15.167:0/2602855275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a50102de0 0x7f9a50100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:22.640 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.639+0000 7f9a55e64640 1 --2- 172.21.15.167:0/2602855275 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f9a3806e050 0x7f9a38070510 unknown :-1 s=CLOSED pgs=3509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:22.640 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.639+0000 7f9a55e64640 1 --2- 172.21.15.167:0/2602855275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a501024c0 0x7f9a50100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:22.640 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.639+0000 7f9a55e64640 1 -- 172.21.15.167:0/2602855275 >> 172.21.15.167:0/2602855275 conn(0x7f9a500fbe70 msgr2=0x7f9a501031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:36:22.640 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.639+0000 7f9a55e64640 1 -- 172.21.15.167:0/2602855275 shutdown_connections 2024-06-18T13:36:22.640 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.639+0000 7f9a55e64640 1 -- 172.21.15.167:0/2602855275 wait complete. 2024-06-18T13:36:22.653 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool set-quota 281aa14d-36f8-4e55-8e76-736d78c61052 max_objects 10 2024-06-18T13:36:22.743 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.741+0000 7fbd79a08640 1 -- 172.21.15.167:0/1282718488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd74069920 msgr2=0x7fbd74105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:22.743 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.741+0000 7fbd79a08640 1 --2- 172.21.15.167:0/1282718488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd74069920 0x7fbd74105ec0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009900 tx=0x7fbd6802bc30 comp rx=0 tx=0).stop 2024-06-18T13:36:22.743 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.741+0000 7fbd79a08640 1 -- 172.21.15.167:0/1282718488 shutdown_connections 2024-06-18T13:36:22.743 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.741+0000 7fbd79a08640 1 --2- 172.21.15.167:0/1282718488 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd74106620 0x7fbd7410e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:22.743 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.741+0000 7fbd79a08640 1 --2- 172.21.15.167:0/1282718488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd74069920 0x7fbd74105ec0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:22.743 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.741+0000 7fbd79a08640 1 --2- 172.21.15.167:0/1282718488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd74068f70 0x7fbd74069350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:22.743 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.741+0000 7fbd79a08640 1 -- 172.21.15.167:0/1282718488 >> 172.21.15.167:0/1282718488 conn(0x7fbd74075f30 msgr2=0x7fbd74076340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:36:22.744 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.741+0000 7fbd79a08640 1 -- 172.21.15.167:0/1282718488 shutdown_connections 2024-06-18T13:36:22.744 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.741+0000 7fbd79a08640 1 -- 172.21.15.167:0/1282718488 wait complete. 2024-06-18T13:36:22.744 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.741+0000 7fbd79a08640 1 Processor -- start 2024-06-18T13:36:22.744 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.741+0000 7fbd79a08640 1 -- start start 2024-06-18T13:36:22.744 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd79a08640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd74068f70 0x7fbd74197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:22.744 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd79a08640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd74106620 0x7fbd741976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:22.744 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd79a08640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd7419ba20 0x7fbd7419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:22.744 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd79a08640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74110ad0 con 0x7fbd7419ba20 2024-06-18T13:36:22.744 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd79a08640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74110950 con 0x7fbd74068f70 2024-06-18T13:36:22.744 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd79a08640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbd74110c50 con 0x7fbd74106620 2024-06-18T13:36:22.744 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd727fc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd74106620 0x7fbd741976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:22.744 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd737fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd7419ba20 0x7fbd7419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:22.744 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd727fc640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd74106620 0x7fbd741976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:56142/0 (socket says 172.21.15.167:56142) 2024-06-18T13:36:22.745 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd727fc640 1 -- 172.21.15.167:0/2312165644 learned_addr learned my addr 172.21.15.167:0/2312165644 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:36:22.745 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd72ffd640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd74068f70 0x7fbd74197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:22.745 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd727fc640 1 -- 172.21.15.167:0/2312165644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd74068f70 msgr2=0x7fbd74197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:22.745 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd727fc640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd74068f70 0x7fbd74197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:22.745 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd727fc640 1 -- 172.21.15.167:0/2312165644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd7419ba20 msgr2=0x7fbd7419bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:22.745 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd727fc640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd7419ba20 0x7fbd7419bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:22.745 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd727fc640 1 -- 172.21.15.167:0/2312165644 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7419c510 con 0x7fbd74106620 2024-06-18T13:36:22.745 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd737fe640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd7419ba20 0x7fbd7419bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:36:22.745 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd72ffd640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd74068f70 0x7fbd74197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T13:36:22.745 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd727fc640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd74106620 0x7fbd741976a0 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7fbd68000c00 tx=0x7fbd68004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:36:22.745 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd78a06640 1 -- 172.21.15.167:0/2312165644 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68043020 con 0x7fbd74106620 2024-06-18T13:36:22.745 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd79a08640 1 -- 172.21.15.167:0/2312165644 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7419c7a0 con 0x7fbd74106620 2024-06-18T13:36:22.745 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd78a06640 1 -- 172.21.15.167:0/2312165644 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fbd68004a30 con 0x7fbd74106620 2024-06-18T13:36:22.746 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd78a06640 1 -- 172.21.15.167:0/2312165644 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68003800 con 0x7fbd74106620 2024-06-18T13:36:22.746 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.742+0000 7fbd79a08640 1 -- 172.21.15.167:0/2312165644 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd741a3ff0 con 0x7fbd74106620 2024-06-18T13:36:22.746 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.744+0000 7fbd79a08640 1 -- 172.21.15.167:0/2312165644 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd741029a0 con 0x7fbd74106620 2024-06-18T13:36:22.746 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.744+0000 7fbd78a06640 1 -- 172.21.15.167:0/2312165644 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7fbd68046020 con 0x7fbd74106620 2024-06-18T13:36:22.747 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.745+0000 7fbd78a06640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbd5c06e050 0x7fbd5c070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:36:22.747 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.745+0000 7fbd78a06640 1 -- 172.21.15.167:0/2312165644 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(779..779 src has 257..779) ==== 9480+0+0 (secure 0 0 0) 0x7fbd680b0ec0 con 0x7fbd74106620 2024-06-18T13:36:22.747 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.745+0000 7fbd72ffd640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbd5c06e050 0x7fbd5c070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:36:22.747 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.745+0000 7fbd72ffd640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbd5c06e050 0x7fbd5c070510 secure :-1 s=READY pgs=3510 cs=0 l=1 rev1=1 crypto rx=0x7fbd64005fd0 tx=0x7fbd64009520 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:36:22.749 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.749+0000 7fbd78a06640 1 -- 172.21.15.167:0/2312165644 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fbd6804d020 con 0x7fbd74106620 2024-06-18T13:36:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:22 smithi179 ceph-mon[33312]: pgmap v1706: 188 pgs: 12 unknown, 176 active+clean; 724 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:22 smithi179 ceph-mon[33312]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:36:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2602855275' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"}]': finished 2024-06-18T13:36:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:22 smithi179 ceph-mon[33312]: osdmap e778: 8 total, 8 up, 8 in 2024-06-18T13:36:22.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:22 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2602855275' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"}]: dispatch 2024-06-18T13:36:22.926 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:22.926+0000 7fbd79a08640 1 -- 172.21.15.167:0/2312165644 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"} v 0) -- 0x7fbd741984b0 con 0x7fbd74106620 2024-06-18T13:36:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:22 smithi167 ceph-mon[37701]: pgmap v1706: 188 pgs: 12 unknown, 176 active+clean; 724 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:22 smithi167 ceph-mon[37701]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:36:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2602855275' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"}]': finished 2024-06-18T13:36:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:22 smithi167 ceph-mon[37701]: osdmap e778: 8 total, 8 up, 8 in 2024-06-18T13:36:22.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:22 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2602855275' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"}]: dispatch 2024-06-18T13:36:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:22 smithi167 ceph-mon[29496]: pgmap v1706: 188 pgs: 12 unknown, 176 active+clean; 724 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:22 smithi167 ceph-mon[29496]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:36:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2602855275' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"}]': finished 2024-06-18T13:36:22.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:22 smithi167 ceph-mon[29496]: osdmap e778: 8 total, 8 up, 8 in 2024-06-18T13:36:22.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:22 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2602855275' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"}]: dispatch 2024-06-18T13:36:23.652 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:23.652+0000 7fbd78a06640 1 -- 172.21.15.167:0/2312165644 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 281aa14d-36f8-4e55-8e76-736d78c61052 v780) ==== 223+0+0 (secure 0 0 0) 0x7fbd6803c350 con 0x7fbd74106620 2024-06-18T13:36:23.757 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:23.757+0000 7fbd79a08640 1 -- 172.21.15.167:0/2312165644 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"} v 0) -- 0x7fbd74069ac0 con 0x7fbd74106620 2024-06-18T13:36:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2602855275' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"}]': finished 2024-06-18T13:36:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:23 smithi167 ceph-mon[37701]: osdmap e779: 8 total, 8 up, 8 in 2024-06-18T13:36:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:23 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2312165644' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:36:23.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:23 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:36:23.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2602855275' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"}]': finished 2024-06-18T13:36:23.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:23 smithi167 ceph-mon[29496]: osdmap e779: 8 total, 8 up, 8 in 2024-06-18T13:36:23.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:23 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2312165644' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:36:23.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:23 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:36:24.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2602855275' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "app": "rados"}]': finished 2024-06-18T13:36:24.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:23 smithi179 ceph-mon[33312]: osdmap e779: 8 total, 8 up, 8 in 2024-06-18T13:36:24.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:23 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2312165644' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:36:24.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:23 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:36:24.654 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:24.653+0000 7fbd78a06640 1 -- 172.21.15.167:0/2312165644 <== mon.2 v2:172.21.15.167:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 281aa14d-36f8-4e55-8e76-736d78c61052 v781) ==== 223+0+0 (secure 0 0 0) 0x7fbd6807da60 con 0x7fbd74106620 2024-06-18T13:36:24.654 INFO:tasks.workunit.client.0.smithi167.stderr:set-quota max_objects = 10 for pool 281aa14d-36f8-4e55-8e76-736d78c61052 2024-06-18T13:36:24.656 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:24.656+0000 7fbd79a08640 1 -- 172.21.15.167:0/2312165644 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbd5c06e050 msgr2=0x7fbd5c070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:24.656 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:24.656+0000 7fbd79a08640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbd5c06e050 0x7fbd5c070510 secure :-1 s=READY pgs=3510 cs=0 l=1 rev1=1 crypto rx=0x7fbd64005fd0 tx=0x7fbd64009520 comp rx=0 tx=0).stop 2024-06-18T13:36:24.657 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:24.656+0000 7fbd79a08640 1 -- 172.21.15.167:0/2312165644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd74106620 msgr2=0x7fbd741976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:36:24.657 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:24.656+0000 7fbd79a08640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd74106620 0x7fbd741976a0 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7fbd68000c00 tx=0x7fbd68004270 comp rx=0 tx=0).stop 2024-06-18T13:36:24.657 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:24.656+0000 7fbd79a08640 1 -- 172.21.15.167:0/2312165644 shutdown_connections 2024-06-18T13:36:24.657 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:24.656+0000 7fbd79a08640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd7419ba20 0x7fbd7419bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:24.657 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:24.656+0000 7fbd79a08640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd74106620 0x7fbd741976a0 unknown :-1 s=CLOSED pgs=2824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:24.657 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:24.656+0000 7fbd79a08640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fbd5c06e050 0x7fbd5c070510 unknown :-1 s=CLOSED pgs=3510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:24.657 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:24.656+0000 7fbd79a08640 1 --2- 172.21.15.167:0/2312165644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd74068f70 0x7fbd74197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:36:24.657 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:24.656+0000 7fbd79a08640 1 -- 172.21.15.167:0/2312165644 >> 172.21.15.167:0/2312165644 conn(0x7fbd74075f30 msgr2=0x7fbd740fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:36:24.657 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:24.656+0000 7fbd79a08640 1 -- 172.21.15.167:0/2312165644 shutdown_connections 2024-06-18T13:36:24.657 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:36:24.656+0000 7fbd79a08640 1 -- 172.21.15.167:0/2312165644 wait complete. 2024-06-18T13:36:24.667 INFO:tasks.workunit.client.0.smithi167.stderr:+ sleep 30 2024-06-18T13:36:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:24 smithi167 ceph-mon[37701]: pgmap v1709: 188 pgs: 12 unknown, 176 active+clean; 724 KiB data, 876 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:36:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]': finished 2024-06-18T13:36:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:24 smithi167 ceph-mon[37701]: osdmap e780: 8 total, 8 up, 8 in 2024-06-18T13:36:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:24 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/2312165644' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:36:24.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:24 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:36:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:24 smithi167 ceph-mon[29496]: pgmap v1709: 188 pgs: 12 unknown, 176 active+clean; 724 KiB data, 876 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:36:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]': finished 2024-06-18T13:36:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:24 smithi167 ceph-mon[29496]: osdmap e780: 8 total, 8 up, 8 in 2024-06-18T13:36:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:24 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/2312165644' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:36:24.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:24 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:36:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:24 smithi179 ceph-mon[33312]: pgmap v1709: 188 pgs: 12 unknown, 176 active+clean; 724 KiB data, 876 MiB used, 714 GiB / 715 GiB avail 2024-06-18T13:36:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]': finished 2024-06-18T13:36:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:24 smithi179 ceph-mon[33312]: osdmap e780: 8 total, 8 up, 8 in 2024-06-18T13:36:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:24 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/2312165644' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:36:25.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:24 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]: dispatch 2024-06-18T13:36:25.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:36:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:36:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:36:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:25 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]': finished 2024-06-18T13:36:25.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:25 smithi167 ceph-mon[37701]: osdmap e781: 8 total, 8 up, 8 in 2024-06-18T13:36:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:25 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]': finished 2024-06-18T13:36:25.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:25 smithi167 ceph-mon[29496]: osdmap e781: 8 total, 8 up, 8 in 2024-06-18T13:36:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:25 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "field": "max_objects", "val": "10"}]': finished 2024-06-18T13:36:26.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:25 smithi179 ceph-mon[33312]: osdmap e781: 8 total, 8 up, 8 in 2024-06-18T13:36:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:26 smithi167 ceph-mon[37701]: pgmap v1712: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-06-18T13:36:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:26 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:26.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:26 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:36:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:26 smithi167 ceph-mon[29496]: pgmap v1712: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-06-18T13:36:26.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:26 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:26.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:26 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:36:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:26 smithi179 ceph-mon[33312]: pgmap v1712: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-06-18T13:36:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:26 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:36:27.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:26 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:36:28.939 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:28 smithi179 ceph-mon[33312]: pgmap v1713: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1003 B/s rd, 4.7 KiB/s wr, 1 op/s 2024-06-18T13:36:28.939 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:28 smithi179 ceph-mon[33312]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:36:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:28 smithi167 ceph-mon[29496]: pgmap v1713: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1003 B/s rd, 4.7 KiB/s wr, 1 op/s 2024-06-18T13:36:28.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:28 smithi167 ceph-mon[29496]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:36:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:28 smithi167 ceph-mon[37701]: pgmap v1713: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1003 B/s rd, 4.7 KiB/s wr, 1 op/s 2024-06-18T13:36:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:28 smithi167 ceph-mon[37701]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-18T13:36:30.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:36:30 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:36:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:30 smithi167 ceph-mon[37701]: pgmap v1714: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 3.9 KiB/s wr, 1 op/s 2024-06-18T13:36:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:30 smithi167 ceph-mon[29496]: pgmap v1714: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 3.9 KiB/s wr, 1 op/s 2024-06-18T13:36:31.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:30 smithi179 ceph-mon[33312]: pgmap v1714: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 3.9 KiB/s wr, 1 op/s 2024-06-18T13:36:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:31 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:31 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:32.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:31 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:32 smithi167 ceph-mon[37701]: pgmap v1715: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-06-18T13:36:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:32 smithi167 ceph-mon[29496]: pgmap v1715: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-06-18T13:36:33.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:32 smithi179 ceph-mon[33312]: pgmap v1715: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-06-18T13:36:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:34 smithi167 ceph-mon[29496]: pgmap v1716: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 563 B/s rd, 0 op/s 2024-06-18T13:36:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:34 smithi167 ceph-mon[37701]: pgmap v1716: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 563 B/s rd, 0 op/s 2024-06-18T13:36:35.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:34 smithi179 ceph-mon[33312]: pgmap v1716: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 563 B/s rd, 0 op/s 2024-06-18T13:36:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:36:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:36:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:36:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:36 smithi167 ceph-mon[29496]: pgmap v1717: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1015 B/s rd, 0 op/s 2024-06-18T13:36:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:36 smithi167 ceph-mon[37701]: pgmap v1717: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1015 B/s rd, 0 op/s 2024-06-18T13:36:37.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:36 smithi179 ceph-mon[33312]: pgmap v1717: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1015 B/s rd, 0 op/s 2024-06-18T13:36:38.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:38 smithi167 ceph-mon[37701]: pgmap v1718: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:38 smithi167 ceph-mon[29496]: pgmap v1718: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:39.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:38 smithi179 ceph-mon[33312]: pgmap v1718: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:40.685 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:36:40 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:36:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:40 smithi167 ceph-mon[37701]: pgmap v1719: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:40 smithi167 ceph-mon[29496]: pgmap v1719: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:41.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:40 smithi179 ceph-mon[33312]: pgmap v1719: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:41 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:41 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:36:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:41 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:41.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:41 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:36:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:41 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:42.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:41 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:36:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:42 smithi167 ceph-mon[37701]: pgmap v1720: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:42 smithi167 ceph-mon[29496]: pgmap v1720: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:43.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:42 smithi179 ceph-mon[33312]: pgmap v1720: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:43.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:43 smithi167 ceph-mon[29496]: pgmap v1721: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:43.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:43 smithi167 ceph-mon[37701]: pgmap v1721: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:44.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:43 smithi179 ceph-mon[33312]: pgmap v1721: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:36:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:36:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:36:46.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:45 smithi179 ceph-mon[33312]: pgmap v1722: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:46.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:45 smithi167 ceph-mon[37701]: pgmap v1722: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:46.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:45 smithi167 ceph-mon[29496]: pgmap v1722: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:48.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:47 smithi179 ceph-mon[33312]: pgmap v1723: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:48.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:47 smithi167 ceph-mon[37701]: pgmap v1723: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:48.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:47 smithi167 ceph-mon[29496]: pgmap v1723: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:50.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:49 smithi179 ceph-mon[33312]: pgmap v1724: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:50.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:49 smithi167 ceph-mon[37701]: pgmap v1724: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:50.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:49 smithi167 ceph-mon[29496]: pgmap v1724: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:50.806 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:36:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:36:51.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:50 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:51.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:50 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:51.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:50 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:36:52.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:51 smithi179 ceph-mon[33312]: pgmap v1725: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:52.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:51 smithi167 ceph-mon[37701]: pgmap v1725: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:52.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:51 smithi167 ceph-mon[29496]: pgmap v1725: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:54.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:53 smithi179 ceph-mon[33312]: pgmap v1726: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:54.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:53 smithi167 ceph-mon[37701]: pgmap v1726: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:54.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:53 smithi167 ceph-mon[29496]: pgmap v1726: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:36:54.669 INFO:tasks.workunit.client.0.smithi167.stderr:++ seq 1 10 2024-06-18T13:36:54.669 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:36:54.669 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 281aa14d-36f8-4e55-8e76-736d78c61052 put obj1 /etc/passwd 2024-06-18T13:36:54.713 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:36:54.713 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 281aa14d-36f8-4e55-8e76-736d78c61052 put obj2 /etc/passwd 2024-06-18T13:36:54.752 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:36:54.752 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 281aa14d-36f8-4e55-8e76-736d78c61052 put obj3 /etc/passwd 2024-06-18T13:36:54.788 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:36:54.788 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 281aa14d-36f8-4e55-8e76-736d78c61052 put obj4 /etc/passwd 2024-06-18T13:36:54.826 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:36:54.826 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 281aa14d-36f8-4e55-8e76-736d78c61052 put obj5 /etc/passwd 2024-06-18T13:36:54.861 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:36:54.861 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 281aa14d-36f8-4e55-8e76-736d78c61052 put obj6 /etc/passwd 2024-06-18T13:36:54.896 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:36:54.896 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 281aa14d-36f8-4e55-8e76-736d78c61052 put obj7 /etc/passwd 2024-06-18T13:36:54.932 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:36:54.932 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 281aa14d-36f8-4e55-8e76-736d78c61052 put obj8 /etc/passwd 2024-06-18T13:36:54.968 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:36:54.968 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 281aa14d-36f8-4e55-8e76-736d78c61052 put obj9 /etc/passwd 2024-06-18T13:36:55.004 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in `seq 1 10` 2024-06-18T13:36:55.004 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 281aa14d-36f8-4e55-8e76-736d78c61052 put obj10 /etc/passwd 2024-06-18T13:36:55.039 INFO:tasks.workunit.client.0.smithi167.stderr:+ sleep 30 2024-06-18T13:36:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:36:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:36:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:36:56.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:55 smithi179 ceph-mon[33312]: pgmap v1727: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:56.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:55 smithi167 ceph-mon[37701]: pgmap v1727: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:56.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:55 smithi167 ceph-mon[29496]: pgmap v1727: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:36:57.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:36:57.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:36:57.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:36:58.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:57 smithi179 ceph-mon[33312]: pgmap v1728: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:58.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:57 smithi167 ceph-mon[29496]: pgmap v1728: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:36:58.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:57 smithi167 ceph-mon[37701]: pgmap v1728: 188 pgs: 188 active+clean; 748 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-18T13:37:00.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:36:59 smithi179 ceph-mon[33312]: pgmap v1729: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-06-18T13:37:00.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:36:59 smithi167 ceph-mon[29496]: pgmap v1729: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-06-18T13:37:00.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:36:59 smithi167 ceph-mon[37701]: pgmap v1729: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-06-18T13:37:00.831 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:37:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:37:01.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:00 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:01.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:00 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:01.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:00 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:02.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:01 smithi179 ceph-mon[33312]: pgmap v1730: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-06-18T13:37:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:01 smithi167 ceph-mon[29496]: pgmap v1730: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-06-18T13:37:02.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:01 smithi167 ceph-mon[37701]: pgmap v1730: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-06-18T13:37:04.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:03 smithi179 ceph-mon[33312]: pgmap v1731: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-06-18T13:37:04.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:03 smithi179 ceph-mon[33312]: pool '281aa14d-36f8-4e55-8e76-736d78c61052' is full (reached quota's max_objects: 10) 2024-06-18T13:37:04.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:03 smithi179 ceph-mon[33312]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-18T13:37:04.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:03 smithi179 ceph-mon[33312]: osdmap e782: 8 total, 8 up, 8 in 2024-06-18T13:37:04.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:03 smithi167 ceph-mon[37701]: pgmap v1731: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-06-18T13:37:04.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:03 smithi167 ceph-mon[37701]: pool '281aa14d-36f8-4e55-8e76-736d78c61052' is full (reached quota's max_objects: 10) 2024-06-18T13:37:04.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:03 smithi167 ceph-mon[37701]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-18T13:37:04.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:03 smithi167 ceph-mon[37701]: osdmap e782: 8 total, 8 up, 8 in 2024-06-18T13:37:04.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:03 smithi167 ceph-mon[29496]: pgmap v1731: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-06-18T13:37:04.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:03 smithi167 ceph-mon[29496]: pool '281aa14d-36f8-4e55-8e76-736d78c61052' is full (reached quota's max_objects: 10) 2024-06-18T13:37:04.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:03 smithi167 ceph-mon[29496]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-18T13:37:04.228 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:03 smithi167 ceph-mon[29496]: osdmap e782: 8 total, 8 up, 8 in 2024-06-18T13:37:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:37:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:37:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:37:06.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:05 smithi179 ceph-mon[33312]: pgmap v1733: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-18T13:37:06.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:05 smithi167 ceph-mon[37701]: pgmap v1733: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-18T13:37:06.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:05 smithi167 ceph-mon[29496]: pgmap v1733: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-18T13:37:08.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:07 smithi179 ceph-mon[33312]: pgmap v1734: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-18T13:37:08.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:07 smithi167 ceph-mon[37701]: pgmap v1734: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-18T13:37:08.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:07 smithi167 ceph-mon[29496]: pgmap v1734: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-18T13:37:10.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:09 smithi179 ceph-mon[33312]: pgmap v1735: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:37:10.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:09 smithi167 ceph-mon[29496]: pgmap v1735: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:37:10.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:09 smithi167 ceph-mon[37701]: pgmap v1735: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:37:10.856 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:37:10 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:37:11.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:10 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:11.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:10 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:11.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:10 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:12.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:11 smithi179 ceph-mon[33312]: pgmap v1736: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:37:12.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:37:12.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:37:12.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:11 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:37:12.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:11 smithi167 ceph-mon[29496]: pgmap v1736: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:37:12.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:37:12.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:37:12.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:11 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:37:12.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:11 smithi167 ceph-mon[37701]: pgmap v1736: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:37:12.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:37:12.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:37:12.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:11 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-18T13:37:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:12 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:37:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:12 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:37:13.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:12 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:37:13.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:12 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:37:13.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:12 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:37:13.477 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:12 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:37:13.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:12 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-18T13:37:13.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:12 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-18T13:37:13.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:12 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' 2024-06-18T13:37:14.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:14 smithi179 ceph-mon[33312]: pgmap v1737: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:37:14.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:14 smithi167 ceph-mon[29496]: pgmap v1737: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:37:14.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:14 smithi167 ceph-mon[37701]: pgmap v1737: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-18T13:37:15.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:37:15 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:37:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:37:16.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:16 smithi179 ceph-mon[33312]: pgmap v1738: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-18T13:37:16.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:16 smithi167 ceph-mon[37701]: pgmap v1738: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-18T13:37:16.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:16 smithi167 ceph-mon[29496]: pgmap v1738: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-18T13:37:18.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:18 smithi179 ceph-mon[33312]: pgmap v1739: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:18.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:18 smithi167 ceph-mon[37701]: pgmap v1739: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:18.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:18 smithi167 ceph-mon[29496]: pgmap v1739: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:20.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:20 smithi179 ceph-mon[33312]: pgmap v1740: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:20.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:37:20 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:37:20.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:20 smithi167 ceph-mon[37701]: pgmap v1740: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:20.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:20 smithi167 ceph-mon[29496]: pgmap v1740: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:21.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:21 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:21.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:21 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:21.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:21 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:22.637 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:22 smithi179 ceph-mon[33312]: pgmap v1741: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:22.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:22 smithi167 ceph-mon[37701]: pgmap v1741: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:22.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:22 smithi167 ceph-mon[29496]: pgmap v1741: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:24.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:24 smithi167 ceph-mon[37701]: pgmap v1742: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:24.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:24 smithi167 ceph-mon[29496]: pgmap v1742: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:24.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:24 smithi179 ceph-mon[33312]: pgmap v1742: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:25.041 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p 2d48a5d0-5bf6-4059-9227-066b6e28e48a put threemore /etc/passwd 2024-06-18T13:37:25.082 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool set-quota 2d48a5d0-5bf6-4059-9227-066b6e28e48a max_bytes 0 2024-06-18T13:37:25.174 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.170+0000 7f9874659640 1 -- 172.21.15.167:0/924265723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f986c1014e0 msgr2=0x7f986c1018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:25.174 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.170+0000 7f9874659640 1 --2- 172.21.15.167:0/924265723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f986c1014e0 0x7f986c1018c0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f9868009900 tx=0x7f986802bc30 comp rx=0 tx=0).stop 2024-06-18T13:37:25.174 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.170+0000 7f9874659640 1 -- 172.21.15.167:0/924265723 shutdown_connections 2024-06-18T13:37:25.174 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.170+0000 7f9874659640 1 --2- 172.21.15.167:0/924265723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f986c106630 0x7f986c10e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:25.174 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.170+0000 7f9874659640 1 --2- 172.21.15.167:0/924265723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f986c101e90 0x7f986c105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:25.175 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.170+0000 7f9874659640 1 --2- 172.21.15.167:0/924265723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f986c1014e0 0x7f986c1018c0 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:25.175 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.170+0000 7f9874659640 1 -- 172.21.15.167:0/924265723 >> 172.21.15.167:0/924265723 conn(0x7f986c0fcf90 msgr2=0x7f986c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:37:25.175 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.170+0000 7f9874659640 1 -- 172.21.15.167:0/924265723 shutdown_connections 2024-06-18T13:37:25.175 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.170+0000 7f9874659640 1 -- 172.21.15.167:0/924265723 wait complete. 2024-06-18T13:37:25.175 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.171+0000 7f9874659640 1 Processor -- start 2024-06-18T13:37:25.175 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.171+0000 7f9874659640 1 -- start start 2024-06-18T13:37:25.175 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.171+0000 7f9874659640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f986c1014e0 0x7f986c1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:25.175 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.171+0000 7f9874659640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f986c106630 0x7f986c197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:25.175 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.171+0000 7f9874659640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f986c19b9b0 0x7f986c19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:25.175 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.171+0000 7f9874659640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f986c110ad0 con 0x7f986c1014e0 2024-06-18T13:37:25.176 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.171+0000 7f9874659640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f986c110950 con 0x7f986c106630 2024-06-18T13:37:25.176 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.171+0000 7f9874659640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f986c110c50 con 0x7f986c19b9b0 2024-06-18T13:37:25.176 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.171+0000 7f98723f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f986c1014e0 0x7f986c1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:25.176 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.171+0000 7f9872bf6640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f986c19b9b0 0x7f986c19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:25.176 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.171+0000 7f98723f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f986c1014e0 0x7f986c1970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38422/0 (socket says 172.21.15.167:38422) 2024-06-18T13:37:25.176 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.171+0000 7f98723f5640 1 -- 172.21.15.167:0/3328167714 learned_addr learned my addr 172.21.15.167:0/3328167714 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:37:25.176 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.171+0000 7f9871bf4640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f986c106630 0x7f986c197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:25.176 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.172+0000 7f98723f5640 1 -- 172.21.15.167:0/3328167714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f986c19b9b0 msgr2=0x7f986c19be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:25.176 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.172+0000 7f98723f5640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f986c19b9b0 0x7f986c19be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:25.176 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.172+0000 7f98723f5640 1 -- 172.21.15.167:0/3328167714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f986c106630 msgr2=0x7f986c197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:25.176 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.172+0000 7f98723f5640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f986c106630 0x7f986c197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:25.176 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.172+0000 7f98723f5640 1 -- 172.21.15.167:0/3328167714 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986c19c4a0 con 0x7f986c1014e0 2024-06-18T13:37:25.176 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.172+0000 7f9872bf6640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f986c19b9b0 0x7f986c19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:37:25.176 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.172+0000 7f9871bf4640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f986c106630 0x7f986c197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-18T13:37:25.177 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.172+0000 7f98723f5640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f986c1014e0 0x7f986c1970f0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f9868000c00 tx=0x7f9868004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:37:25.177 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.172+0000 7f98637fe640 1 -- 172.21.15.167:0/3328167714 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868043020 con 0x7f986c1014e0 2024-06-18T13:37:25.177 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.172+0000 7f98637fe640 1 -- 172.21.15.167:0/3328167714 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f9868004a30 con 0x7f986c1014e0 2024-06-18T13:37:25.177 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.172+0000 7f9874659640 1 -- 172.21.15.167:0/3328167714 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986c19c730 con 0x7f986c1014e0 2024-06-18T13:37:25.177 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.172+0000 7f98637fe640 1 -- 172.21.15.167:0/3328167714 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868003890 con 0x7f986c1014e0 2024-06-18T13:37:25.177 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.172+0000 7f9874659640 1 -- 172.21.15.167:0/3328167714 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986c1a3f50 con 0x7f986c1014e0 2024-06-18T13:37:25.177 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.173+0000 7f9874659640 1 -- 172.21.15.167:0/3328167714 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f986c1018c0 con 0x7f986c1014e0 2024-06-18T13:37:25.178 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.174+0000 7f98637fe640 1 -- 172.21.15.167:0/3328167714 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f9868046020 con 0x7f986c1014e0 2024-06-18T13:37:25.178 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.174+0000 7f98637fe640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f985806e050 0x7f9858070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:25.178 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.174+0000 7f98637fe640 1 -- 172.21.15.167:0/3328167714 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(782..782 src has 257..782) ==== 9480+0+0 (secure 0 0 0) 0x7f98680b1200 con 0x7f986c1014e0 2024-06-18T13:37:25.178 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.174+0000 7f9871bf4640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f985806e050 0x7f9858070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:25.178 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.174+0000 7f98637fe640 1 -- 172.21.15.167:0/3328167714 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=783}) -- 0x7f985807a370 con 0x7f986c1014e0 2024-06-18T13:37:25.178 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.175+0000 7f9871bf4640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f985806e050 0x7f9858070510 secure :-1 s=READY pgs=3522 cs=0 l=1 rev1=1 crypto rx=0x7f986c0ff0d0 tx=0x7f985c005f90 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:37:25.180 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.179+0000 7f98637fe640 1 -- 172.21.15.167:0/3328167714 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f986804d020 con 0x7f986c1014e0 2024-06-18T13:37:25.357 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.356+0000 7f9874659640 1 -- 172.21.15.167:0/3328167714 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"} v 0) -- 0x7f986c101e90 con 0x7f986c1014e0 2024-06-18T13:37:25.467 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:37:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:37:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:37:25.474 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.473+0000 7f98637fe640 1 -- 172.21.15.167:0/3328167714 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a v783) ==== 217+0+0 (secure 0 0 0) 0x7f986807dda0 con 0x7f986c1014e0 2024-06-18T13:37:25.475 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.474+0000 7f98637fe640 1 -- 172.21.15.167:0/3328167714 <== mon.0 v2:172.21.15.167:3300/0 8 ==== osd_map(783..783 src has 257..783) ==== 628+0+0 (secure 0 0 0) 0x7f98680e58c0 con 0x7f986c1014e0 2024-06-18T13:37:25.475 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.474+0000 7f98637fe640 1 -- 172.21.15.167:0/3328167714 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=784}) -- 0x7f985807b2d0 con 0x7f986c1014e0 2024-06-18T13:37:25.583 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:25.582+0000 7f9874659640 1 -- 172.21.15.167:0/3328167714 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"} v 0) -- 0x7f986c1a4120 con 0x7f986c1014e0 2024-06-18T13:37:25.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:25 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3328167714' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:37:25.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:25 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3328167714' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:37:25.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:25 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3328167714' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:37:26.480 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.479+0000 7f98637fe640 1 -- 172.21.15.167:0/3328167714 <== mon.0 v2:172.21.15.167:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a v784) ==== 217+0+0 (secure 0 0 0) 0x7f9868081a50 con 0x7f986c1014e0 2024-06-18T13:37:26.481 INFO:tasks.workunit.client.0.smithi167.stderr:set-quota max_bytes = 0 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a 2024-06-18T13:37:26.481 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.479+0000 7f98637fe640 1 -- 172.21.15.167:0/3328167714 <== mon.0 v2:172.21.15.167:3300/0 10 ==== osd_map(784..784 src has 257..784) ==== 628+0+0 (secure 0 0 0) 0x7f986807b740 con 0x7f986c1014e0 2024-06-18T13:37:26.481 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.479+0000 7f98637fe640 1 -- 172.21.15.167:0/3328167714 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=785}) -- 0x7f985807b400 con 0x7f986c1014e0 2024-06-18T13:37:26.484 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.482+0000 7f9874659640 1 -- 172.21.15.167:0/3328167714 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f985806e050 msgr2=0x7f9858070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:26.484 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.482+0000 7f9874659640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f985806e050 0x7f9858070510 secure :-1 s=READY pgs=3522 cs=0 l=1 rev1=1 crypto rx=0x7f986c0ff0d0 tx=0x7f985c005f90 comp rx=0 tx=0).stop 2024-06-18T13:37:26.484 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.482+0000 7f9874659640 1 -- 172.21.15.167:0/3328167714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f986c1014e0 msgr2=0x7f986c1970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:26.484 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.482+0000 7f9874659640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f986c1014e0 0x7f986c1970f0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f9868000c00 tx=0x7f9868004270 comp rx=0 tx=0).stop 2024-06-18T13:37:26.484 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.483+0000 7f9874659640 1 -- 172.21.15.167:0/3328167714 shutdown_connections 2024-06-18T13:37:26.484 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.483+0000 7f9874659640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f986c19b9b0 0x7f986c19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:26.484 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.483+0000 7f9874659640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f985806e050 0x7f9858070510 unknown :-1 s=CLOSED pgs=3522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:26.484 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.483+0000 7f9874659640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f986c106630 0x7f986c197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:26.484 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.483+0000 7f9874659640 1 --2- 172.21.15.167:0/3328167714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f986c1014e0 0x7f986c1970f0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:26.484 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.483+0000 7f9874659640 1 -- 172.21.15.167:0/3328167714 >> 172.21.15.167:0/3328167714 conn(0x7f986c0fcf90 msgr2=0x7f986c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:37:26.485 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.483+0000 7f9874659640 1 -- 172.21.15.167:0/3328167714 shutdown_connections 2024-06-18T13:37:26.485 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.483+0000 7f9874659640 1 -- 172.21.15.167:0/3328167714 wait complete. 2024-06-18T13:37:26.496 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool set-quota 2d48a5d0-5bf6-4059-9227-066b6e28e48a max_objects 0 2024-06-18T13:37:26.584 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.581+0000 7f99c8e86640 1 -- 172.21.15.167:0/1668287958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c4100580 msgr2=0x7f99c41009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:26.585 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.581+0000 7f99c8e86640 1 --2- 172.21.15.167:0/1668287958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c4100580 0x7f99c41009e0 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009890 tx=0x7f99b802bb70 comp rx=0 tx=0).stop 2024-06-18T13:37:26.585 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.581+0000 7f99c8e86640 1 -- 172.21.15.167:0/1668287958 shutdown_connections 2024-06-18T13:37:26.585 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.581+0000 7f99c8e86640 1 --2- 172.21.15.167:0/1668287958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c4100f20 0x7f99c410d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:26.585 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.581+0000 7f99c8e86640 1 --2- 172.21.15.167:0/1668287958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c4100580 0x7f99c41009e0 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:26.585 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.581+0000 7f99c8e86640 1 --2- 172.21.15.167:0/1668287958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99c41051b0 0x7f99c4105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:26.585 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.581+0000 7f99c8e86640 1 -- 172.21.15.167:0/1668287958 >> 172.21.15.167:0/1668287958 conn(0x7f99c4075f30 msgr2=0x7f99c4076340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:37:26.585 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.581+0000 7f99c8e86640 1 -- 172.21.15.167:0/1668287958 shutdown_connections 2024-06-18T13:37:26.585 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.581+0000 7f99c8e86640 1 -- 172.21.15.167:0/1668287958 wait complete. 2024-06-18T13:37:26.585 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c8e86640 1 Processor -- start 2024-06-18T13:37:26.585 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c8e86640 1 -- start start 2024-06-18T13:37:26.585 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c8e86640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c4100580 0x7f99c4197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:26.585 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c8e86640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c4100f20 0x7f99c41975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:26.586 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c8e86640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99c41051b0 0x7f99c419b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:26.586 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c8e86640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99c410f9b0 con 0x7f99c41051b0 2024-06-18T13:37:26.586 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c8e86640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f99c410f830 con 0x7f99c4100f20 2024-06-18T13:37:26.586 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c8e86640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f99c410fb30 con 0x7f99c4100580 2024-06-18T13:37:26.586 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c259c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c4100580 0x7f99c4197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:26.586 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c259c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c4100580 0x7f99c4197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:60540/0 (socket says 172.21.15.167:60540) 2024-06-18T13:37:26.586 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99c41051b0 0x7f99c419b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:26.586 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c259c640 1 -- 172.21.15.167:0/3140975811 learned_addr learned my addr 172.21.15.167:0/3140975811 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:37:26.586 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c259c640 1 -- 172.21.15.167:0/3140975811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c4100f20 msgr2=0x7f99c41975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:26.586 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c1d9b640 1 --2- 172.21.15.167:0/3140975811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c4100f20 0x7f99c41975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:26.586 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c259c640 1 --2- 172.21.15.167:0/3140975811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c4100f20 0x7f99c41975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:26.586 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c259c640 1 -- 172.21.15.167:0/3140975811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99c41051b0 msgr2=0x7f99c419b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:26.587 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c259c640 1 --2- 172.21.15.167:0/3140975811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99c41051b0 0x7f99c419b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:26.587 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c259c640 1 -- 172.21.15.167:0/3140975811 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c419c070 con 0x7f99c4100580 2024-06-18T13:37:26.587 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.582+0000 7f99c2d9d640 1 --2- 172.21.15.167:0/3140975811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99c41051b0 0x7f99c419b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:37:26.587 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.583+0000 7f99c259c640 1 --2- 172.21.15.167:0/3140975811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c4100580 0x7f99c4197080 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f99b400e9c0 tx=0x7f99b400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:37:26.587 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.583+0000 7f99b37fe640 1 -- 172.21.15.167:0/3140975811 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b400cde0 con 0x7f99c4100580 2024-06-18T13:37:26.587 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.583+0000 7f99c8e86640 1 -- 172.21.15.167:0/3140975811 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c419c360 con 0x7f99c4100580 2024-06-18T13:37:26.587 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.583+0000 7f99b37fe640 1 -- 172.21.15.167:0/3140975811 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f99b40035f0 con 0x7f99c4100580 2024-06-18T13:37:26.587 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.583+0000 7f99b37fe640 1 -- 172.21.15.167:0/3140975811 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b40044a0 con 0x7f99c4100580 2024-06-18T13:37:26.587 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.583+0000 7f99c8e86640 1 -- 172.21.15.167:0/3140975811 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c41a3bf0 con 0x7f99c4100580 2024-06-18T13:37:26.588 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.584+0000 7f99c8e86640 1 -- 172.21.15.167:0/3140975811 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c41018b0 con 0x7f99c4100580 2024-06-18T13:37:26.588 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.584+0000 7f99b37fe640 1 -- 172.21.15.167:0/3140975811 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f99b4004000 con 0x7f99c4100580 2024-06-18T13:37:26.588 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.585+0000 7f99b37fe640 1 --2- 172.21.15.167:0/3140975811 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f99ac06e050 0x7f99ac070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:26.588 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.585+0000 7f99c1d9b640 1 --2- 172.21.15.167:0/3140975811 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f99ac06e050 0x7f99ac070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:26.588 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.585+0000 7f99b37fe640 1 -- 172.21.15.167:0/3140975811 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(784..784 src has 257..784) ==== 9480+0+0 (secure 0 0 0) 0x7f99b408f190 con 0x7f99c4100580 2024-06-18T13:37:26.588 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.585+0000 7f99b37fe640 1 -- 172.21.15.167:0/3140975811 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=785}) -- 0x7f99ac07a3d0 con 0x7f99c4100580 2024-06-18T13:37:26.588 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.585+0000 7f99c1d9b640 1 --2- 172.21.15.167:0/3140975811 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f99ac06e050 0x7f99ac070510 secure :-1 s=READY pgs=3523 cs=0 l=1 rev1=1 crypto rx=0x7f99c40766c0 tx=0x7f99b8000bc0 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:37:26.590 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.589+0000 7f99b37fe640 1 -- 172.21.15.167:0/3140975811 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f99b405bd30 con 0x7f99c4100580 2024-06-18T13:37:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:26 smithi167 ceph-mon[29496]: pgmap v1743: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:26.727 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3328167714' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]': finished 2024-06-18T13:37:26.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:26 smithi167 ceph-mon[29496]: osdmap e783: 8 total, 8 up, 8 in 2024-06-18T13:37:26.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:26 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3328167714' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:37:26.728 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:26 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:37:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:26 smithi167 ceph-mon[37701]: pgmap v1743: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:26.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3328167714' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]': finished 2024-06-18T13:37:26.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:26 smithi167 ceph-mon[37701]: osdmap e783: 8 total, 8 up, 8 in 2024-06-18T13:37:26.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:26 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3328167714' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:37:26.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:26 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:37:26.766 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:26.765+0000 7f99c8e86640 1 -- 172.21.15.167:0/3140975811 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"} v 0) -- 0x7f99c41009e0 con 0x7f99c4100580 2024-06-18T13:37:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:26 smithi179 ceph-mon[33312]: pgmap v1743: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3328167714' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]': finished 2024-06-18T13:37:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:26 smithi179 ceph-mon[33312]: osdmap e783: 8 total, 8 up, 8 in 2024-06-18T13:37:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:26 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3328167714' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-18T13:37:26.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:26 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:37:27.486 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:27.484+0000 7f99b37fe640 1 -- 172.21.15.167:0/3140975811 <== mon.2 v2:172.21.15.167:3301/0 7 ==== osd_map(785..785 src has 257..785) ==== 628+0+0 (secure 0 0 0) 0x7f99b40596d0 con 0x7f99c4100580 2024-06-18T13:37:27.486 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:27.484+0000 7f99b37fe640 1 -- 172.21.15.167:0/3140975811 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=786}) -- 0x7f99ac07af80 con 0x7f99c4100580 2024-06-18T13:37:27.490 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:27.489+0000 7f99b37fe640 1 -- 172.21.15.167:0/3140975811 <== mon.2 v2:172.21.15.167:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a v785) ==== 221+0+0 (secure 0 0 0) 0x7f99b405f9e0 con 0x7f99c4100580 2024-06-18T13:37:27.596 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:27.595+0000 7f99c8e86640 1 -- 172.21.15.167:0/3140975811 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"} v 0) -- 0x7f99c41a3e20 con 0x7f99c4100580 2024-06-18T13:37:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3328167714' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]': finished 2024-06-18T13:37:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:27 smithi179 ceph-mon[33312]: osdmap e784: 8 total, 8 up, 8 in 2024-06-18T13:37:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:27 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3140975811' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:37:27.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:27 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:37:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3328167714' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]': finished 2024-06-18T13:37:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:27 smithi167 ceph-mon[37701]: osdmap e784: 8 total, 8 up, 8 in 2024-06-18T13:37:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:27 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3140975811' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:37:27.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:27 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:37:27.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3328167714' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_bytes", "val": "0"}]': finished 2024-06-18T13:37:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:27 smithi167 ceph-mon[29496]: osdmap e784: 8 total, 8 up, 8 in 2024-06-18T13:37:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:27 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3140975811' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:37:27.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:27 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:37:28.488 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.486+0000 7f99b37fe640 1 -- 172.21.15.167:0/3140975811 <== mon.2 v2:172.21.15.167:3301/0 9 ==== osd_map(786..786 src has 257..786) ==== 628+0+0 (secure 0 0 0) 0x7f99b408c890 con 0x7f99c4100580 2024-06-18T13:37:28.488 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.486+0000 7f99b37fe640 1 -- 172.21.15.167:0/3140975811 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=787}) -- 0x7f99ac07b3b0 con 0x7f99c4100580 2024-06-18T13:37:28.492 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.491+0000 7f99b37fe640 1 -- 172.21.15.167:0/3140975811 <== mon.2 v2:172.21.15.167:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a v786) ==== 221+0+0 (secure 0 0 0) 0x7f99b4064210 con 0x7f99c4100580 2024-06-18T13:37:28.492 INFO:tasks.workunit.client.0.smithi167.stderr:set-quota max_objects = 0 for pool 2d48a5d0-5bf6-4059-9227-066b6e28e48a 2024-06-18T13:37:28.496 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.495+0000 7f99c8e86640 1 -- 172.21.15.167:0/3140975811 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f99ac06e050 msgr2=0x7f99ac070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:28.496 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.495+0000 7f99c8e86640 1 --2- 172.21.15.167:0/3140975811 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f99ac06e050 0x7f99ac070510 secure :-1 s=READY pgs=3523 cs=0 l=1 rev1=1 crypto rx=0x7f99c40766c0 tx=0x7f99b8000bc0 comp rx=0 tx=0).stop 2024-06-18T13:37:28.496 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.495+0000 7f99c8e86640 1 -- 172.21.15.167:0/3140975811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c4100580 msgr2=0x7f99c4197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:28.496 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.495+0000 7f99c8e86640 1 --2- 172.21.15.167:0/3140975811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c4100580 0x7f99c4197080 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f99b400e9c0 tx=0x7f99b400ee80 comp rx=0 tx=0).stop 2024-06-18T13:37:28.496 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.495+0000 7f99c8e86640 1 -- 172.21.15.167:0/3140975811 shutdown_connections 2024-06-18T13:37:28.496 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.495+0000 7f99c8e86640 1 --2- 172.21.15.167:0/3140975811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99c41051b0 0x7f99c419b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:28.496 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.495+0000 7f99c8e86640 1 --2- 172.21.15.167:0/3140975811 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f99ac06e050 0x7f99ac070510 unknown :-1 s=CLOSED pgs=3523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:28.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.495+0000 7f99c8e86640 1 --2- 172.21.15.167:0/3140975811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c4100f20 0x7f99c41975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:28.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.495+0000 7f99c8e86640 1 --2- 172.21.15.167:0/3140975811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c4100580 0x7f99c4197080 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:28.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.495+0000 7f99c8e86640 1 -- 172.21.15.167:0/3140975811 >> 172.21.15.167:0/3140975811 conn(0x7f99c4075f30 msgr2=0x7f99c41043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:37:28.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.495+0000 7f99c8e86640 1 -- 172.21.15.167:0/3140975811 shutdown_connections 2024-06-18T13:37:28.497 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:28.495+0000 7f99c8e86640 1 -- 172.21.15.167:0/3140975811 wait complete. 2024-06-18T13:37:28.507 INFO:tasks.workunit.client.0.smithi167.stderr:+ sleep 30 2024-06-18T13:37:28.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:28 smithi179 ceph-mon[33312]: pgmap v1746: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:37:28.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]': finished 2024-06-18T13:37:28.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:28 smithi179 ceph-mon[33312]: osdmap e785: 8 total, 8 up, 8 in 2024-06-18T13:37:28.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:28 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/3140975811' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:37:28.789 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:28 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:37:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:28 smithi167 ceph-mon[37701]: pgmap v1746: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:37:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]': finished 2024-06-18T13:37:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:28 smithi167 ceph-mon[37701]: osdmap e785: 8 total, 8 up, 8 in 2024-06-18T13:37:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:28 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/3140975811' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:37:28.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:28 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:37:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:28 smithi167 ceph-mon[29496]: pgmap v1746: 188 pgs: 188 active+clean; 866 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-18T13:37:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]': finished 2024-06-18T13:37:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:28 smithi167 ceph-mon[29496]: osdmap e785: 8 total, 8 up, 8 in 2024-06-18T13:37:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:28 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/3140975811' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:37:28.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:28 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]: dispatch 2024-06-18T13:37:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:29 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]': finished 2024-06-18T13:37:29.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:29 smithi179 ceph-mon[33312]: osdmap e786: 8 total, 8 up, 8 in 2024-06-18T13:37:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:29 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]': finished 2024-06-18T13:37:29.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:29 smithi167 ceph-mon[37701]: osdmap e786: 8 total, 8 up, 8 in 2024-06-18T13:37:29.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:29 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "field": "max_objects", "val": "0"}]': finished 2024-06-18T13:37:29.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:29 smithi167 ceph-mon[29496]: osdmap e786: 8 total, 8 up, 8 in 2024-06-18T13:37:30.890 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:37:30 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:37:30.890 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:30 smithi179 ceph-mon[33312]: pgmap v1749: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-06-18T13:37:30.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:30 smithi167 ceph-mon[37701]: pgmap v1749: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-06-18T13:37:30.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:30 smithi167 ceph-mon[29496]: pgmap v1749: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-06-18T13:37:31.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:31 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:31.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:31 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:31.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:31 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:32.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:32 smithi179 ceph-mon[33312]: pgmap v1750: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 970 B/s rd, 2.3 KiB/s wr, 1 op/s 2024-06-18T13:37:32.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:32 smithi167 ceph-mon[37701]: pgmap v1750: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 970 B/s rd, 2.3 KiB/s wr, 1 op/s 2024-06-18T13:37:32.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:32 smithi167 ceph-mon[29496]: pgmap v1750: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 970 B/s rd, 2.3 KiB/s wr, 1 op/s 2024-06-18T13:37:34.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:34 smithi179 ceph-mon[33312]: pgmap v1751: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 816 B/s rd, 1.9 KiB/s wr, 0 op/s 2024-06-18T13:37:34.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:34 smithi167 ceph-mon[37701]: pgmap v1751: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 816 B/s rd, 1.9 KiB/s wr, 0 op/s 2024-06-18T13:37:34.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:34 smithi167 ceph-mon[29496]: pgmap v1751: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 816 B/s rd, 1.9 KiB/s wr, 0 op/s 2024-06-18T13:37:35.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:37:35 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:37:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:37:36.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:36 smithi179 ceph-mon[33312]: pgmap v1752: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-06-18T13:37:36.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:36 smithi167 ceph-mon[29496]: pgmap v1752: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-06-18T13:37:36.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:36 smithi167 ceph-mon[37701]: pgmap v1752: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-06-18T13:37:38.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:38 smithi179 ceph-mon[33312]: pgmap v1753: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-06-18T13:37:38.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:38 smithi167 ceph-mon[29496]: pgmap v1753: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-06-18T13:37:38.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:38 smithi167 ceph-mon[37701]: pgmap v1753: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-06-18T13:37:40.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:40 smithi179 ceph-mon[33312]: pgmap v1754: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 997 B/s rd, 0 op/s 2024-06-18T13:37:40.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:37:40 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:37:40.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:40 smithi167 ceph-mon[29496]: pgmap v1754: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 997 B/s rd, 0 op/s 2024-06-18T13:37:40.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:40 smithi167 ceph-mon[37701]: pgmap v1754: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 997 B/s rd, 0 op/s 2024-06-18T13:37:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:41 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:41.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:41 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:37:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:41 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:41.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:41 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:37:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:41 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:41.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:41 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:37:42.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:42 smithi179 ceph-mon[33312]: pgmap v1755: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:42.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:42 smithi167 ceph-mon[37701]: pgmap v1755: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:42.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:42 smithi167 ceph-mon[29496]: pgmap v1755: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:44.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:44 smithi179 ceph-mon[33312]: pgmap v1756: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:44.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:44 smithi167 ceph-mon[37701]: pgmap v1756: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:44.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:44 smithi167 ceph-mon[29496]: pgmap v1756: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:45.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:37:45 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:37:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:37:46.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:46 smithi179 ceph-mon[33312]: pgmap v1757: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:46.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:46 smithi167 ceph-mon[37701]: pgmap v1757: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:46.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:46 smithi167 ceph-mon[29496]: pgmap v1757: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:48.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:48 smithi179 ceph-mon[33312]: pgmap v1758: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:48.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:48 smithi167 ceph-mon[29496]: pgmap v1758: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:48.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:48 smithi167 ceph-mon[37701]: pgmap v1758: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-18T13:37:50.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:50 smithi179 ceph-mon[33312]: pgmap v1759: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:50.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:37:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:37:50.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:50 smithi167 ceph-mon[29496]: pgmap v1759: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:50.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:50 smithi167 ceph-mon[37701]: pgmap v1759: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:51.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:51 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:51.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:51 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:51.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:51 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:37:52.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:52 smithi179 ceph-mon[33312]: pgmap v1760: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:37:52.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:52 smithi167 ceph-mon[29496]: pgmap v1760: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:37:52.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:52 smithi167 ceph-mon[37701]: pgmap v1760: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:37:54.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:54 smithi179 ceph-mon[33312]: pgmap v1761: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:37:54.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:54 smithi167 ceph-mon[29496]: pgmap v1761: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:37:54.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:54 smithi167 ceph-mon[37701]: pgmap v1761: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:37:55.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:37:55 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:37:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:37:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:56 smithi179 ceph-mon[33312]: pgmap v1762: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:56.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:56 smithi179 ceph-mon[33312]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:37:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:56 smithi167 ceph-mon[37701]: pgmap v1762: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:56.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:56 smithi167 ceph-mon[37701]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:37:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:56 smithi167 ceph-mon[29496]: pgmap v1762: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:37:56.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:56 smithi167 ceph-mon[29496]: from='mgr.24461 172.21.15.167:0/311903193' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-18T13:37:58.509 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool delete 2d48a5d0-5bf6-4059-9227-066b6e28e48a 2d48a5d0-5bf6-4059-9227-066b6e28e48a --yes-i-really-really-mean-it 2024-06-18T13:37:58.603 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.603+0000 7fb899f5c640 1 -- 172.21.15.167:0/225183438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8941051b0 msgr2=0x7fb894105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:58.603 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.603+0000 7fb899f5c640 1 --2- 172.21.15.167:0/225183438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8941051b0 0x7fb894105590 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7fb884009890 tx=0x7fb88402bb70 comp rx=0 tx=0).stop 2024-06-18T13:37:58.604 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.603+0000 7fb899f5c640 1 -- 172.21.15.167:0/225183438 shutdown_connections 2024-06-18T13:37:58.604 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.603+0000 7fb899f5c640 1 --2- 172.21.15.167:0/225183438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb894100f20 0x7fb89410d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:58.604 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.603+0000 7fb899f5c640 1 --2- 172.21.15.167:0/225183438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb894100580 0x7fb8941009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:58.604 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.603+0000 7fb899f5c640 1 --2- 172.21.15.167:0/225183438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8941051b0 0x7fb894105590 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:58.604 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.603+0000 7fb899f5c640 1 -- 172.21.15.167:0/225183438 >> 172.21.15.167:0/225183438 conn(0x7fb894075f30 msgr2=0x7fb894076340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:37:58.604 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.603+0000 7fb899f5c640 1 -- 172.21.15.167:0/225183438 shutdown_connections 2024-06-18T13:37:58.604 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.603+0000 7fb899f5c640 1 -- 172.21.15.167:0/225183438 wait complete. 2024-06-18T13:37:58.604 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.603+0000 7fb899f5c640 1 Processor -- start 2024-06-18T13:37:58.606 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb899f5c640 1 -- start start 2024-06-18T13:37:58.606 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb899f5c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb894100580 0x7fb8941970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:58.607 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb899f5c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb894100f20 0x7fb894197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:58.607 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb899f5c640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8941051b0 0x7fb89419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:58.607 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb899f5c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb89410f990 con 0x7fb894100580 2024-06-18T13:37:58.607 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb899f5c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb89410f810 con 0x7fb894100f20 2024-06-18T13:37:58.607 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb899f5c640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb89410fb10 con 0x7fb8941051b0 2024-06-18T13:37:58.607 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb8937fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb894100580 0x7fb8941970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:58.607 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb893fff640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8941051b0 0x7fb89419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:58.607 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb8937fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb894100580 0x7fb8941970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55386/0 (socket says 172.21.15.167:55386) 2024-06-18T13:37:58.607 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb8937fe640 1 -- 172.21.15.167:0/955520022 learned_addr learned my addr 172.21.15.167:0/955520022 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:37:58.607 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb892ffd640 1 --2- 172.21.15.167:0/955520022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb894100f20 0x7fb894197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:58.607 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb8937fe640 1 -- 172.21.15.167:0/955520022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8941051b0 msgr2=0x7fb89419b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:58.608 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb8937fe640 1 --2- 172.21.15.167:0/955520022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8941051b0 0x7fb89419b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:58.608 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb8937fe640 1 -- 172.21.15.167:0/955520022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb894100f20 msgr2=0x7fb894197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:58.608 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb8937fe640 1 --2- 172.21.15.167:0/955520022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb894100f20 0x7fb894197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:58.608 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb8937fe640 1 -- 172.21.15.167:0/955520022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89419c0d0 con 0x7fb894100580 2024-06-18T13:37:58.608 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.604+0000 7fb8937fe640 1 --2- 172.21.15.167:0/955520022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb894100580 0x7fb8941970e0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fb884035fd0 tx=0x7fb8840040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:37:58.608 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.605+0000 7fb890ff9640 1 -- 172.21.15.167:0/955520022 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884041020 con 0x7fb894100580 2024-06-18T13:37:58.608 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.605+0000 7fb899f5c640 1 -- 172.21.15.167:0/955520022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89419c360 con 0x7fb894100580 2024-06-18T13:37:58.608 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.605+0000 7fb890ff9640 1 -- 172.21.15.167:0/955520022 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fb88402c990 con 0x7fb894100580 2024-06-18T13:37:58.608 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.605+0000 7fb890ff9640 1 -- 172.21.15.167:0/955520022 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884039040 con 0x7fb894100580 2024-06-18T13:37:58.608 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.605+0000 7fb899f5c640 1 -- 172.21.15.167:0/955520022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8941a3c50 con 0x7fb894100580 2024-06-18T13:37:58.608 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.606+0000 7fb899f5c640 1 -- 172.21.15.167:0/955520022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8941018b0 con 0x7fb894100580 2024-06-18T13:37:58.608 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.606+0000 7fb890ff9640 1 -- 172.21.15.167:0/955520022 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7fb884044020 con 0x7fb894100580 2024-06-18T13:37:58.609 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.607+0000 7fb890ff9640 1 --2- 172.21.15.167:0/955520022 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb87c06e050 0x7fb87c070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:58.609 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.607+0000 7fb890ff9640 1 -- 172.21.15.167:0/955520022 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(786..786 src has 257..786) ==== 9480+0+0 (secure 0 0 0) 0x7fb8840b0680 con 0x7fb894100580 2024-06-18T13:37:58.609 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.607+0000 7fb892ffd640 1 --2- 172.21.15.167:0/955520022 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb87c06e050 0x7fb87c070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:58.609 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.607+0000 7fb890ff9640 1 -- 172.21.15.167:0/955520022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=787}) -- 0x7fb87c07a370 con 0x7fb894100580 2024-06-18T13:37:58.609 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.607+0000 7fb892ffd640 1 --2- 172.21.15.167:0/955520022 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb87c06e050 0x7fb87c070510 secure :-1 s=READY pgs=3524 cs=0 l=1 rev1=1 crypto rx=0x7fb894104ab0 tx=0x7fb888009260 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:37:58.611 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.611+0000 7fb890ff9640 1 -- 172.21.15.167:0/955520022 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7fb88404b020 con 0x7fb894100580 2024-06-18T13:37:58.790 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:58.790+0000 7fb899f5c640 1 -- 172.21.15.167:0/955520022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pool2": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb894198430 con 0x7fb894100580 2024-06-18T13:37:58.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:58 smithi179 ceph-mon[33312]: pgmap v1763: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:37:58.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:58 smithi167 ceph-mon[37701]: pgmap v1763: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:37:58.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:58 smithi167 ceph-mon[29496]: pgmap v1763: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-18T13:37:59.570 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.569+0000 7fb890ff9640 1 -- 172.21.15.167:0/955520022 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pool2": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "yes_i_really_really_mean_it": true}]=0 pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' removed v787) ==== 248+0+0 (secure 0 0 0) 0x7fb884080ed0 con 0x7fb894100580 2024-06-18T13:37:59.571 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.571+0000 7fb890ff9640 1 -- 172.21.15.167:0/955520022 <== mon.0 v2:172.21.15.167:3300/0 8 ==== osd_map(787..787 src has 257..787) ==== 296+0+0 (secure 0 0 0) 0x7fb8840e48c0 con 0x7fb894100580 2024-06-18T13:37:59.571 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.571+0000 7fb890ff9640 1 -- 172.21.15.167:0/955520022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=788}) -- 0x7fb87c07ae70 con 0x7fb894100580 2024-06-18T13:37:59.686 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.686+0000 7fb899f5c640 1 -- 172.21.15.167:0/955520022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pool2": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb894061e90 con 0x7fb894100580 2024-06-18T13:37:59.687 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.687+0000 7fb890ff9640 1 -- 172.21.15.167:0/955520022 <== mon.0 v2:172.21.15.167:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pool2": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "yes_i_really_really_mean_it": true}]=0 pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' does not exist v787) ==== 255+0+0 (secure 0 0 0) 0x7fb8840b0060 con 0x7fb894100580 2024-06-18T13:37:59.687 INFO:tasks.workunit.client.0.smithi167.stderr:pool '2d48a5d0-5bf6-4059-9227-066b6e28e48a' does not exist 2024-06-18T13:37:59.689 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.689+0000 7fb899f5c640 1 -- 172.21.15.167:0/955520022 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb87c06e050 msgr2=0x7fb87c070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:59.689 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.689+0000 7fb899f5c640 1 --2- 172.21.15.167:0/955520022 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb87c06e050 0x7fb87c070510 secure :-1 s=READY pgs=3524 cs=0 l=1 rev1=1 crypto rx=0x7fb894104ab0 tx=0x7fb888009260 comp rx=0 tx=0).stop 2024-06-18T13:37:59.689 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.689+0000 7fb899f5c640 1 -- 172.21.15.167:0/955520022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb894100580 msgr2=0x7fb8941970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:59.689 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.689+0000 7fb899f5c640 1 --2- 172.21.15.167:0/955520022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb894100580 0x7fb8941970e0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fb884035fd0 tx=0x7fb8840040c0 comp rx=0 tx=0).stop 2024-06-18T13:37:59.690 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.689+0000 7fb899f5c640 1 -- 172.21.15.167:0/955520022 shutdown_connections 2024-06-18T13:37:59.690 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.689+0000 7fb899f5c640 1 --2- 172.21.15.167:0/955520022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8941051b0 0x7fb89419b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:59.690 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.689+0000 7fb899f5c640 1 --2- 172.21.15.167:0/955520022 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7fb87c06e050 0x7fb87c070510 unknown :-1 s=CLOSED pgs=3524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:59.690 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.689+0000 7fb899f5c640 1 --2- 172.21.15.167:0/955520022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb894100f20 0x7fb894197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:59.690 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.689+0000 7fb899f5c640 1 --2- 172.21.15.167:0/955520022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb894100580 0x7fb8941970e0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:59.690 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.689+0000 7fb899f5c640 1 -- 172.21.15.167:0/955520022 >> 172.21.15.167:0/955520022 conn(0x7fb894075f30 msgr2=0x7fb894068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:37:59.690 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.689+0000 7fb899f5c640 1 -- 172.21.15.167:0/955520022 shutdown_connections 2024-06-18T13:37:59.690 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.689+0000 7fb899f5c640 1 -- 172.21.15.167:0/955520022 wait complete. 2024-06-18T13:37:59.698 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool delete 281aa14d-36f8-4e55-8e76-736d78c61052 281aa14d-36f8-4e55-8e76-736d78c61052 --yes-i-really-really-mean-it 2024-06-18T13:37:59.784 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.781+0000 7f3170af4640 1 -- 172.21.15.167:0/2928176494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c107910 msgr2=0x7f316c10e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:59.784 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.781+0000 7f3170af4640 1 --2- 172.21.15.167:0/2928176494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c107910 0x7f316c10e260 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f3164009cd0 tx=0x7f316402c180 comp rx=0 tx=0).stop 2024-06-18T13:37:59.785 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.781+0000 7f3170af4640 1 -- 172.21.15.167:0/2928176494 shutdown_connections 2024-06-18T13:37:59.785 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.781+0000 7f3170af4640 1 --2- 172.21.15.167:0/2928176494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c107910 0x7f316c10e260 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:59.785 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.781+0000 7f3170af4640 1 --2- 172.21.15.167:0/2928176494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f316c100970 0x7f316c1071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:59.785 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.781+0000 7f3170af4640 1 --2- 172.21.15.167:0/2928176494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f316c100050 0x7f316c100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:59.785 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.781+0000 7f3170af4640 1 -- 172.21.15.167:0/2928176494 >> 172.21.15.167:0/2928176494 conn(0x7f316c0756a0 msgr2=0x7f316c075ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:37:59.785 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.781+0000 7f3170af4640 1 -- 172.21.15.167:0/2928176494 shutdown_connections 2024-06-18T13:37:59.785 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.781+0000 7f3170af4640 1 -- 172.21.15.167:0/2928176494 wait complete. 2024-06-18T13:37:59.785 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.782+0000 7f3170af4640 1 Processor -- start 2024-06-18T13:37:59.785 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.782+0000 7f3170af4640 1 -- start start 2024-06-18T13:37:59.785 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.782+0000 7f3170af4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c100050 0x7f316c197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:59.785 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.782+0000 7f3170af4640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f316c100970 0x7f316c197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:59.786 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.782+0000 7f3170af4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f316c107910 0x7f316c19b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:59.786 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.782+0000 7f3170af4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f316c110b70 con 0x7f316c100050 2024-06-18T13:37:59.786 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.782+0000 7f3170af4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f316c1109f0 con 0x7f316c107910 2024-06-18T13:37:59.786 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.782+0000 7f3170af4640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f316c110cf0 con 0x7f316c100970 2024-06-18T13:37:59.786 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.782+0000 7f316a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c100050 0x7f316c197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:59.786 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.782+0000 7f3169d9b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f316c100970 0x7f316c197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:59.786 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.782+0000 7f316a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c100050 0x7f316c197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55394/0 (socket says 172.21.15.167:55394) 2024-06-18T13:37:59.786 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.783+0000 7f316a59c640 1 -- 172.21.15.167:0/1988520065 learned_addr learned my addr 172.21.15.167:0/1988520065 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-18T13:37:59.786 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.782+0000 7f3169d9b640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f316c100970 0x7f316c197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:56268/0 (socket says 172.21.15.167:56268) 2024-06-18T13:37:59.786 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.783+0000 7f316ad9d640 1 --2- 172.21.15.167:0/1988520065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f316c107910 0x7f316c19b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:59.786 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.783+0000 7f3169d9b640 1 -- 172.21.15.167:0/1988520065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f316c107910 msgr2=0x7f316c19b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:59.786 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.783+0000 7f3169d9b640 1 --2- 172.21.15.167:0/1988520065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f316c107910 0x7f316c19b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:59.787 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.783+0000 7f3169d9b640 1 -- 172.21.15.167:0/1988520065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c100050 msgr2=0x7f316c197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:37:59.787 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.783+0000 7f3169d9b640 1 --2- 172.21.15.167:0/1988520065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c100050 0x7f316c197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:37:59.787 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.783+0000 7f3169d9b640 1 -- 172.21.15.167:0/1988520065 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316c19c120 con 0x7f316c100970 2024-06-18T13:37:59.787 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.783+0000 7f316a59c640 1 --2- 172.21.15.167:0/1988520065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c100050 0x7f316c197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-18T13:37:59.787 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.783+0000 7f3169d9b640 1 --2- 172.21.15.167:0/1988520065 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f316c100970 0x7f316c197670 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f315c00e9f0 tx=0x7f315c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:37:59.787 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.783+0000 7f315b7fe640 1 -- 172.21.15.167:0/1988520065 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c00cd80 con 0x7f316c100970 2024-06-18T13:37:59.787 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.783+0000 7f315b7fe640 1 -- 172.21.15.167:0/1988520065 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f315c0035f0 con 0x7f316c100970 2024-06-18T13:37:59.787 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.783+0000 7f3170af4640 1 -- 172.21.15.167:0/1988520065 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316c19c410 con 0x7f316c100970 2024-06-18T13:37:59.787 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.783+0000 7f315b7fe640 1 -- 172.21.15.167:0/1988520065 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c0044a0 con 0x7f316c100970 2024-06-18T13:37:59.787 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.783+0000 7f3170af4640 1 -- 172.21.15.167:0/1988520065 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f316c1a3ca0 con 0x7f316c100970 2024-06-18T13:37:59.787 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.784+0000 7f3170af4640 1 -- 172.21.15.167:0/1988520065 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f316c100430 con 0x7f316c100970 2024-06-18T13:37:59.788 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.785+0000 7f315b7fe640 1 -- 172.21.15.167:0/1988520065 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 22) ==== 92175+0+0 (secure 0 0 0) 0x7f315c004000 con 0x7f316c100970 2024-06-18T13:37:59.788 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.785+0000 7f315b7fe640 1 --2- 172.21.15.167:0/1988520065 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f315406e050 0x7f3154070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-18T13:37:59.788 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.785+0000 7f315b7fe640 1 -- 172.21.15.167:0/1988520065 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(787..787 src has 257..787) ==== 9092+0+0 (secure 0 0 0) 0x7f315c08ef30 con 0x7f316c100970 2024-06-18T13:37:59.788 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.785+0000 7f316a59c640 1 --2- 172.21.15.167:0/1988520065 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f315406e050 0x7f3154070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-18T13:37:59.789 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.785+0000 7f315b7fe640 1 -- 172.21.15.167:0/1988520065 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=788}) -- 0x7f3154079b50 con 0x7f316c100970 2024-06-18T13:37:59.789 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.786+0000 7f316a59c640 1 --2- 172.21.15.167:0/1988520065 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f315406e050 0x7f3154070510 secure :-1 s=READY pgs=3525 cs=0 l=1 rev1=1 crypto rx=0x7f3160005fd0 tx=0x7f316000b480 comp rx=0 tx=0).ready entity=mgr.24461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-18T13:37:59.789 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.789+0000 7f315b7fe640 1 -- 172.21.15.167:0/1988520065 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186542 (secure 0 0 0) 0x7f315c05bc50 con 0x7f316c100970 2024-06-18T13:37:59.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:37:59 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/955520022' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pool2": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:37:59.963 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:37:59.962+0000 7f3170af4640 1 -- 172.21.15.167:0/1988520065 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true} v 0) -- 0x7f316c1a3ed0 con 0x7f316c100970 2024-06-18T13:37:59.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:37:59 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/955520022' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pool2": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:37:59.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:37:59 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/955520022' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pool2": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:00.572 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.572+0000 7f315b7fe640 1 -- 172.21.15.167:0/1988520065 <== mon.2 v2:172.21.15.167:3301/0 7 ==== osd_map(788..788 src has 257..788) ==== 296+0+0 (secure 0 0 0) 0x7f315c0595f0 con 0x7f316c100970 2024-06-18T13:38:00.572 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.572+0000 7f315b7fe640 1 -- 172.21.15.167:0/1988520065 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=789}) -- 0x7f315407a380 con 0x7f316c100970 2024-06-18T13:38:00.575 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.575+0000 7f315b7fe640 1 -- 172.21.15.167:0/1988520065 <== mon.2 v2:172.21.15.167:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]=0 pool '281aa14d-36f8-4e55-8e76-736d78c61052' removed v788) ==== 248+0+0 (secure 0 0 0) 0x7f315c05f900 con 0x7f316c100970 2024-06-18T13:38:00.687 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.687+0000 7f3170af4640 1 -- 172.21.15.167:0/1988520065 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true} v 0) -- 0x7f316c061fb0 con 0x7f316c100970 2024-06-18T13:38:00.688 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.688+0000 7f315b7fe640 1 -- 172.21.15.167:0/1988520065 <== mon.2 v2:172.21.15.167:3301/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]=0 pool '281aa14d-36f8-4e55-8e76-736d78c61052' does not exist v788) ==== 255+0+0 (secure 0 0 0) 0x7f315c064130 con 0x7f316c100970 2024-06-18T13:38:00.688 INFO:tasks.workunit.client.0.smithi167.stderr:pool '281aa14d-36f8-4e55-8e76-736d78c61052' does not exist 2024-06-18T13:38:00.690 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.690+0000 7f3170af4640 1 -- 172.21.15.167:0/1988520065 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f315406e050 msgr2=0x7f3154070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:38:00.691 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.690+0000 7f3170af4640 1 --2- 172.21.15.167:0/1988520065 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f315406e050 0x7f3154070510 secure :-1 s=READY pgs=3525 cs=0 l=1 rev1=1 crypto rx=0x7f3160005fd0 tx=0x7f316000b480 comp rx=0 tx=0).stop 2024-06-18T13:38:00.691 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.690+0000 7f3170af4640 1 -- 172.21.15.167:0/1988520065 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f316c100970 msgr2=0x7f316c197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-18T13:38:00.691 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.690+0000 7f3170af4640 1 --2- 172.21.15.167:0/1988520065 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f316c100970 0x7f316c197670 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f315c00e9f0 tx=0x7f315c00eeb0 comp rx=0 tx=0).stop 2024-06-18T13:38:00.691 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.690+0000 7f3170af4640 1 -- 172.21.15.167:0/1988520065 shutdown_connections 2024-06-18T13:38:00.691 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.690+0000 7f3170af4640 1 --2- 172.21.15.167:0/1988520065 >> [v2:172.21.15.167:6800/2104657861,v1:172.21.15.167:6801/2104657861] conn(0x7f315406e050 0x7f3154070510 unknown :-1 s=CLOSED pgs=3525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:38:00.691 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.690+0000 7f3170af4640 1 --2- 172.21.15.167:0/1988520065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f316c107910 0x7f316c19b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:38:00.691 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.690+0000 7f3170af4640 1 --2- 172.21.15.167:0/1988520065 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f316c100970 0x7f316c197670 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:38:00.691 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.690+0000 7f3170af4640 1 --2- 172.21.15.167:0/1988520065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f316c100050 0x7f316c197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-18T13:38:00.691 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.690+0000 7f3170af4640 1 -- 172.21.15.167:0/1988520065 >> 172.21.15.167:0/1988520065 conn(0x7f316c0756a0 msgr2=0x7f316c1055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-18T13:38:00.691 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.690+0000 7f3170af4640 1 -- 172.21.15.167:0/1988520065 shutdown_connections 2024-06-18T13:38:00.691 INFO:tasks.workunit.client.0.smithi167.stderr:2024-06-18T13:38:00.690+0000 7f3170af4640 1 -- 172.21.15.167:0/1988520065 wait complete. 2024-06-18T13:38:00.699 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo OK 2024-06-18T13:38:00.700 INFO:tasks.workunit.client.0.smithi167.stdout:OK 2024-06-18T13:38:00.701 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-18T13:38:00.701 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-18T13:38:00.768 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-06-18T13:38:00.768 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-18T13:38:00.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:38:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug there is no tcmu-runner data available 2024-06-18T13:38:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:00 smithi179 ceph-mon[33312]: pgmap v1764: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:38:00.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/955520022' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pool2": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "yes_i_really_really_mean_it": true}]': finished 2024-06-18T13:38:00.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:00 smithi179 ceph-mon[33312]: osdmap e787: 8 total, 8 up, 8 in 2024-06-18T13:38:00.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/955520022' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pool2": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:00.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:00 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1988520065' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:00.888 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:00 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:00 smithi167 ceph-mon[37701]: pgmap v1764: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:38:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/955520022' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pool2": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "yes_i_really_really_mean_it": true}]': finished 2024-06-18T13:38:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:00 smithi167 ceph-mon[37701]: osdmap e787: 8 total, 8 up, 8 in 2024-06-18T13:38:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/955520022' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pool2": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:00 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1988520065' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:00.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:00 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:00.977 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:00 smithi167 ceph-mon[29496]: pgmap v1764: 188 pgs: 188 active+clean; 878 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-18T13:38:00.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/955520022' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pool2": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "yes_i_really_really_mean_it": true}]': finished 2024-06-18T13:38:00.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:00 smithi167 ceph-mon[29496]: osdmap e787: 8 total, 8 up, 8 in 2024-06-18T13:38:00.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/955520022' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "pool2": "2d48a5d0-5bf6-4059-9227-066b6e28e48a", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:00.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:00 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1988520065' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:00.978 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:00 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:01.097 DEBUG:teuthology.parallel:result is None 2024-06-18T13:38:01.097 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-18T13:38:01.123 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-18T13:38:01.123 DEBUG:teuthology.orchestra.run.smithi167:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-06-18T13:38:01.211 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-06-18T13:38:01.211 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-18T13:38:01.220 INFO:tasks.cephadm:Teardown begin 2024-06-18T13:38:01.220 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-18T13:38:01.245 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-18T13:38:01.279 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-18T13:38:01.279 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-18T13:38:01.301 DEBUG:teuthology.orchestra.run.smithi179:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-18T13:38:01.337 INFO:tasks.cephadm:Stopping all daemons... 2024-06-18T13:38:01.337 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-18T13:38:01.337 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.a 2024-06-18T13:38:01.687 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-mon[37701]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:38:01.687 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-mon[37701]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-18T13:38:01.687 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]': finished 2024-06-18T13:38:01.687 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-mon[37701]: osdmap e788: 8 total, 8 up, 8 in 2024-06-18T13:38:01.687 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-mon[37701]: from='client.? 172.21.15.167:0/1988520065' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:01.687 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-mon[37701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:01.688 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:01 smithi167 systemd[1]: Stopping Ceph mon.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:38:01.688 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-mon[29496]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:38:01.688 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-mon[29496]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-18T13:38:01.688 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]': finished 2024-06-18T13:38:01.688 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-mon[29496]: osdmap e788: 8 total, 8 up, 8 in 2024-06-18T13:38:01.688 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-mon[29496]: from='client.? 172.21.15.167:0/1988520065' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:01.688 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-mon[29496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:01 smithi179 ceph-mon[33312]: from='client.24424 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-18T13:38:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:01 smithi179 ceph-mon[33312]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-18T13:38:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]': finished 2024-06-18T13:38:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:01 smithi179 ceph-mon[33312]: osdmap e788: 8 total, 8 up, 8 in 2024-06-18T13:38:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:01 smithi179 ceph-mon[33312]: from='client.? 172.21.15.167:0/1988520065' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:01.887 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:01 smithi179 ceph-mon[33312]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "281aa14d-36f8-4e55-8e76-736d78c61052", "pool2": "281aa14d-36f8-4e55-8e76-736d78c61052", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-18T13:38:01.973 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:38:01] ENGINE Bus STOPPING 2024-06-18T13:38:01.973 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a[29492]: 2024-06-18T13:38:01.686+0000 7fb19fe43640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-18T13:38:01.973 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:01 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a[29492]: 2024-06-18T13:38:01.686+0000 7fb19fe43640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-18T13:38:01.973 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:01 smithi167 podman[151508]: 2024-06-18 13:38:01.737907231 +0000 UTC m=+0.204713205 container died 4e45951ebad496c4427e9aa68caa028740d64a8e343098ac6f001775737670d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240612, ceph=True, RELEASE=squid-3781e3a, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-18T13:38:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:02 smithi167 podman[151508]: 2024-06-18 13:38:02.172880272 +0000 UTC m=+0.639686248 container cleanup 4e45951ebad496c4427e9aa68caa028740d64a8e343098ac6f001775737670d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-3781e3a, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-18T13:38:02.227 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:02 smithi167 bash[151508]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a 2024-06-18T13:38:02.227 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:38:02] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-18T13:38:02.227 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:38:02] ENGINE Bus STOPPED 2024-06-18T13:38:02.228 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:38:02] ENGINE Bus STARTING 2024-06-18T13:38:02.228 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:38:02] ENGINE Serving on http://:::9283 2024-06-18T13:38:02.228 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:38:02] ENGINE Bus STARTED 2024-06-18T13:38:02.553 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.a.service' 2024-06-18T13:38:02.574 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:02 smithi167 podman[151520]: 2024-06-18 13:38:02.324412188 +0000 UTC m=+0.578791442 container remove 4e45951ebad496c4427e9aa68caa028740d64a8e343098ac6f001775737670d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-a, RELEASE=squid-3781e3a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0) 2024-06-18T13:38:02.574 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:02 smithi167 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-18T13:38:02.574 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:02 smithi167 systemd[1]: Stopped Ceph mon.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:38:02.574 INFO:journalctl@ceph.mon.a.smithi167.stdout:Jun 18 13:38:02 smithi167 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.a.service: Consumed 2min 9.251s CPU time. 2024-06-18T13:38:02.590 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:38:02.590 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-18T13:38:02.591 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-06-18T13:38:02.591 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.c 2024-06-18T13:38:02.918 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:02 smithi167 systemd[1]: Stopping Ceph mon.c for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:38:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-c[37697]: 2024-06-18T13:38:02.918+0000 7f2409dc9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-18T13:38:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-c[37697]: 2024-06-18T13:38:02.918+0000 7f2409dc9640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-06-18T13:38:03.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:03 smithi167 podman[151640]: 2024-06-18 13:38:03.093064853 +0000 UTC m=+0.270924300 container died de3cc70740a845bc58ce314fbe054ccc5071896eddc5c83a101ebf1acfb521e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-c, GIT_CLEAN=True, RELEASE=squid-3781e3a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, ceph=True) 2024-06-18T13:38:03.227 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:02 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:38:02] ENGINE Bus STOPPING 2024-06-18T13:38:03.227 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:03 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:38:03] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-18T13:38:03.227 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:03 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:38:03] ENGINE Bus STOPPED 2024-06-18T13:38:03.228 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:03 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:38:03] ENGINE Bus STARTING 2024-06-18T13:38:03.703 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:03 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:38:03] ENGINE Serving on http://:::9283 2024-06-18T13:38:03.704 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:03 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: [18/Jun/2024:13:38:03] ENGINE Bus STARTED 2024-06-18T13:38:03.704 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:03 smithi167 podman[151640]: 2024-06-18 13:38:03.453017619 +0000 UTC m=+0.630877054 container cleanup de3cc70740a845bc58ce314fbe054ccc5071896eddc5c83a101ebf1acfb521e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3781e3a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-06-18T13:38:03.704 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 18 13:38:03 smithi167 bash[151640]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-c 2024-06-18T13:38:03.883 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.c.service' 2024-06-18T13:38:03.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:38:03.918 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-06-18T13:38:03.918 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-06-18T13:38:03.918 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.b 2024-06-18T13:38:04.387 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:03 smithi179 systemd[1]: Stopping Ceph mon.b for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:38:04.854 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:04 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-b[33295]: 2024-06-18T13:38:04.478+0000 7fde20b42640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-18T13:38:04.854 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:04 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-b[33295]: 2024-06-18T13:38:04.478+0000 7fde20b42640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-06-18T13:38:04.854 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:04 smithi179 podman[74949]: 2024-06-18 13:38:04.57058287 +0000 UTC m=+0.445573479 container died 2d859870fb8d98193ea2f052ba33d183ec0cd11096cb82fe58fd1a570ec3e03a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-b, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240612, RELEASE=squid-3781e3a, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-18T13:38:05.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:05 smithi179 podman[74949]: 2024-06-18 13:38:05.019960737 +0000 UTC m=+0.894951343 container cleanup 2d859870fb8d98193ea2f052ba33d183ec0cd11096cb82fe58fd1a570ec3e03a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240612, RELEASE=squid-3781e3a) 2024-06-18T13:38:05.137 INFO:journalctl@ceph.mon.b.smithi179.stdout:Jun 18 13:38:05 smithi179 bash[74949]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mon-b 2024-06-18T13:38:05.477 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: ::ffff:172.21.15.179 - - [18/Jun/2024:13:38:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-18T13:38:05.484 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mon.b.service' 2024-06-18T13:38:05.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:38:05.534 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-06-18T13:38:05.534 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-18T13:38:05.534 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mgr.y 2024-06-18T13:38:05.857 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:05 smithi167 systemd[1]: Stopping Ceph mgr.y for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:38:06.149 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:38:05.857+0000 7f73988c8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-18T13:38:06.150 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:05 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y[29724]: 2024-06-18T13:38:05.857+0000 7f73988c8640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-06-18T13:38:06.150 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:05 smithi167 podman[151772]: 2024-06-18 13:38:05.889042504 +0000 UTC m=+0.150849429 container died c2b474c3156f158284c9371bd88226d874c28ac90d9e5a95b8dd8ca08fdd575d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-3781e3a, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-18T13:38:06.727 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:06 smithi167 podman[151772]: 2024-06-18 13:38:06.41646831 +0000 UTC m=+0.678275233 container cleanup c2b474c3156f158284c9371bd88226d874c28ac90d9e5a95b8dd8ca08fdd575d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240612, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3781e3a) 2024-06-18T13:38:06.727 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:06 smithi167 bash[151772]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y 2024-06-18T13:38:06.727 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Jun 18 13:38:06 smithi167 podman[151787]: 2024-06-18 13:38:06.635325275 +0000 UTC m=+0.743378179 container remove c2b474c3156f158284c9371bd88226d874c28ac90d9e5a95b8dd8ca08fdd575d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3781e3a592387d16e8020f5db99cdee4198f6d00, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-y, GIT_CLEAN=True, RELEASE=squid-3781e3a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-18T13:38:06.900 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mgr.y.service' 2024-06-18T13:38:06.934 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:38:06.935 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-18T13:38:06.935 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-18T13:38:06.935 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mgr.x 2024-06-18T13:38:07.262 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:38:06 smithi179 systemd[1]: Stopping Ceph mgr.x for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:38:07.534 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:38:07 smithi179 podman[75074]: 2024-06-18 13:38:07.2626743 +0000 UTC m=+0.132560129 container died 6cf236d55942243383614e0ebdc00e09998604c301efd450e0f9b083bb0f94c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3781e3a, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240612, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-06-18T13:38:07.792 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:38:07 smithi179 podman[75074]: 2024-06-18 13:38:07.608484487 +0000 UTC m=+0.478370328 container cleanup 6cf236d55942243383614e0ebdc00e09998604c301efd450e0f9b083bb0f94c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3781e3a) 2024-06-18T13:38:07.793 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:38:07 smithi179 bash[75074]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x 2024-06-18T13:38:08.029 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mgr.x.service' 2024-06-18T13:38:08.050 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:38:07 smithi179 podman[75088]: 2024-06-18 13:38:07.793094748 +0000 UTC m=+0.521370914 container remove 6cf236d55942243383614e0ebdc00e09998604c301efd450e0f9b083bb0f94c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-3781e3a, org.label-schema.build-date=20240612, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-18T13:38:08.051 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:38:07 smithi179 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-06-18T13:38:08.051 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:38:08 smithi179 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mgr.x.service: Failed with result 'exit-code'. 2024-06-18T13:38:08.051 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:38:08 smithi179 systemd[1]: Stopped Ceph mgr.x for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:38:08.051 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Jun 18 13:38:08 smithi179 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@mgr.x.service: Consumed 13.095s CPU time. 2024-06-18T13:38:08.064 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:38:08.064 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-18T13:38:08.064 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-18T13:38:08.064 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.0 2024-06-18T13:38:08.362 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:38:08 smithi167 systemd[1]: Stopping Ceph osd.0 for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:38:08.727 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:38:08 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0[41556]: 2024-06-18T13:38:08.362+0000 7f1632143640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-18T13:38:08.727 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:38:08 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0[41556]: 2024-06-18T13:38:08.362+0000 7f1632143640 -1 osd.0 788 *** Got signal Terminated *** 2024-06-18T13:38:08.727 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:38:08 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0[41556]: 2024-06-18T13:38:08.362+0000 7f1632143640 -1 osd.0 788 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-18T13:38:14.172 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:38:13 smithi167 podman[151896]: 2024-06-18 13:38:13.905319611 +0000 UTC m=+5.643825201 container died cef459c7a131ca2ea05f8844d18fedf3198bd7f1ebb90b5cfd5973182ecca479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-3781e3a, ceph=True) 2024-06-18T13:38:14.439 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:38:14 smithi167 podman[151896]: 2024-06-18 13:38:14.246410517 +0000 UTC m=+5.984916107 container cleanup cef459c7a131ca2ea05f8844d18fedf3198bd7f1ebb90b5cfd5973182ecca479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=squid-3781e3a) 2024-06-18T13:38:14.439 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:38:14 smithi167 bash[151896]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0 2024-06-18T13:38:14.727 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:38:14 smithi167 podman[151911]: 2024-06-18 13:38:14.439445536 +0000 UTC m=+0.524943793 container remove cef459c7a131ca2ea05f8844d18fedf3198bd7f1ebb90b5cfd5973182ecca479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240612, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, RELEASE=squid-3781e3a, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-18T13:38:15.109 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:38:14 smithi167 podman[152005]: 2024-06-18 13:38:14.758650384 +0000 UTC m=+0.017724422 image pull af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314 2024-06-18T13:38:15.109 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:38:14 smithi167 podman[152005]: 2024-06-18 13:38:14.882086378 +0000 UTC m=+0.141160443 container create 61e41e65e6943e42d82062a0217da0b5a1c97ce463c96d684281a8da04e8f4ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240612, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, RELEASE=squid-3781e3a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-18T13:38:15.109 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:38:15 smithi167 podman[152005]: 2024-06-18 13:38:15.109723185 +0000 UTC m=+0.368797218 container init 61e41e65e6943e42d82062a0217da0b5a1c97ce463c96d684281a8da04e8f4ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-3781e3a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612) 2024-06-18T13:38:15.367 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:38:15 smithi167 podman[152005]: 2024-06-18 13:38:15.113508085 +0000 UTC m=+0.372582111 container start 61e41e65e6943e42d82062a0217da0b5a1c97ce463c96d684281a8da04e8f4ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0-deactivate, RELEASE=squid-3781e3a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240612, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-18T13:38:15.367 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:38:15 smithi167 podman[152005]: 2024-06-18 13:38:15.210062212 +0000 UTC m=+0.469136235 container attach 61e41e65e6943e42d82062a0217da0b5a1c97ce463c96d684281a8da04e8f4ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-3781e3a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-18T13:38:15.646 INFO:journalctl@ceph.osd.0.smithi167.stdout:Jun 18 13:38:15 smithi167 podman[152005]: 2024-06-18 13:38:15.367370469 +0000 UTC m=+0.626444500 container died 61e41e65e6943e42d82062a0217da0b5a1c97ce463c96d684281a8da04e8f4ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=squid-3781e3a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-18T13:38:15.928 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.0.service' 2024-06-18T13:38:15.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:38:15.968 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-18T13:38:15.968 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-18T13:38:15.969 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.1 2024-06-18T13:38:16.227 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:38:16 smithi167 systemd[1]: Stopping Ceph osd.1 for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:38:16.727 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:38:16 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-1[47148]: 2024-06-18T13:38:16.334+0000 7f08685ee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-18T13:38:16.727 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:38:16 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-1[47148]: 2024-06-18T13:38:16.334+0000 7f08685ee640 -1 osd.1 788 *** Got signal Terminated *** 2024-06-18T13:38:16.727 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:38:16 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-1[47148]: 2024-06-18T13:38:16.334+0000 7f08685ee640 -1 osd.1 788 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-18T13:38:21.978 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:38:21 smithi167 podman[152112]: 2024-06-18 13:38:21.711887648 +0000 UTC m=+5.515146139 container died 41e09ad56ec9d7903e5e69af2725b0fac4f3a1793f846944fee2d5c32fa8e332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-1, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-3781e3a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240612) 2024-06-18T13:38:22.360 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:38:22 smithi167 podman[152112]: 2024-06-18 13:38:22.125303738 +0000 UTC m=+5.928562230 container cleanup 41e09ad56ec9d7903e5e69af2725b0fac4f3a1793f846944fee2d5c32fa8e332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-1, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3781e3a, io.buildah.version=1.36.0) 2024-06-18T13:38:22.360 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:38:22 smithi167 bash[152112]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-1 2024-06-18T13:38:22.727 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:38:22 smithi167 podman[152140]: 2024-06-18 13:38:22.360143823 +0000 UTC m=+0.645174009 container remove 41e09ad56ec9d7903e5e69af2725b0fac4f3a1793f846944fee2d5c32fa8e332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=squid-3781e3a, ceph=True, org.label-schema.build-date=20240612) 2024-06-18T13:38:23.054 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:38:22 smithi167 podman[152229]: 2024-06-18 13:38:22.702790119 +0000 UTC m=+0.017102391 image pull af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314 2024-06-18T13:38:23.055 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:38:22 smithi167 podman[152229]: 2024-06-18 13:38:22.811011055 +0000 UTC m=+0.125323313 container create 10fbae4d5cb2737d69e8976047812b0720787f03a5774ac3b8426610b0d2fcf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240612, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=squid-3781e3a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-18T13:38:23.055 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:38:23 smithi167 podman[152229]: 2024-06-18 13:38:23.055220628 +0000 UTC m=+0.369532901 container init 10fbae4d5cb2737d69e8976047812b0720787f03a5774ac3b8426610b0d2fcf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3781e3a, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-06-18T13:38:23.316 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:38:23 smithi167 podman[152229]: 2024-06-18 13:38:23.058535999 +0000 UTC m=+0.372848257 container start 10fbae4d5cb2737d69e8976047812b0720787f03a5774ac3b8426610b0d2fcf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3781e3a) 2024-06-18T13:38:23.317 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:38:23 smithi167 podman[152229]: 2024-06-18 13:38:23.138546217 +0000 UTC m=+0.452858475 container attach 10fbae4d5cb2737d69e8976047812b0720787f03a5774ac3b8426610b0d2fcf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3781e3a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-18T13:38:23.624 INFO:journalctl@ceph.osd.1.smithi167.stdout:Jun 18 13:38:23 smithi167 podman[152229]: 2024-06-18 13:38:23.317149336 +0000 UTC m=+0.631461598 container died 10fbae4d5cb2737d69e8976047812b0720787f03a5774ac3b8426610b0d2fcf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, RELEASE=squid-3781e3a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-18T13:38:23.962 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.1.service' 2024-06-18T13:38:24.000 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:38:24.000 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-18T13:38:24.000 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-18T13:38:24.000 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.2 2024-06-18T13:38:24.227 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:38:24 smithi167 systemd[1]: Stopping Ceph osd.2 for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:38:24.637 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:38:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:38:24.229Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.167:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.167:8765: connect: connection refused" 2024-06-18T13:38:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:38:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:38:24.229Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.167:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.167:8765: connect: connection refused" 2024-06-18T13:38:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:38:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:38:24.232Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.167:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.167:8765: connect: connection refused" 2024-06-18T13:38:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:38:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:38:24.232Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.167:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.167:8765: connect: connection refused" 2024-06-18T13:38:24.638 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:38:24 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:38:24.232Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.167:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.167:8765: connect: connection refused" 2024-06-18T13:38:24.727 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:38:24 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2[52740]: 2024-06-18T13:38:24.319+0000 7f087fe60640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-18T13:38:24.727 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:38:24 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2[52740]: 2024-06-18T13:38:24.319+0000 7f087fe60640 -1 osd.2 788 *** Got signal Terminated *** 2024-06-18T13:38:24.727 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:38:24 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2[52740]: 2024-06-18T13:38:24.319+0000 7f087fe60640 -1 osd.2 788 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-18T13:38:29.969 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:38:29 smithi167 podman[152331]: 2024-06-18 13:38:29.64832214 +0000 UTC m=+5.427812645 container died 4d611f9ff95f0863fd54f9fcc8981c897778628ebf1b593c97e29057ca4f61a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240612, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-3781e3a, maintainer=Guillaume Abrioux ) 2024-06-18T13:38:29.970 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:38:29 smithi167 podman[152331]: 2024-06-18 13:38:29.969832641 +0000 UTC m=+5.749323143 container cleanup 4d611f9ff95f0863fd54f9fcc8981c897778628ebf1b593c97e29057ca4f61a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3781e3a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612) 2024-06-18T13:38:30.227 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:38:29 smithi167 bash[152331]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2 2024-06-18T13:38:30.227 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:38:30 smithi167 podman[152345]: 2024-06-18 13:38:30.154437176 +0000 UTC m=+0.501267733 container remove 4d611f9ff95f0863fd54f9fcc8981c897778628ebf1b593c97e29057ca4f61a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3781e3a) 2024-06-18T13:38:30.857 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:38:30 smithi167 podman[152438]: 2024-06-18 13:38:30.498946486 +0000 UTC m=+0.017179382 image pull af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314 2024-06-18T13:38:30.857 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:38:30 smithi167 podman[152438]: 2024-06-18 13:38:30.622065875 +0000 UTC m=+0.140298823 container create cfe8643b1bbb6b1e8a06f6c4737251b4da44382ff44bf86669a0d3233e0993b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3781e3a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-18T13:38:30.857 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:38:30 smithi167 podman[152438]: 2024-06-18 13:38:30.857907722 +0000 UTC m=+0.376140622 container init cfe8643b1bbb6b1e8a06f6c4737251b4da44382ff44bf86669a0d3233e0993b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3781e3a, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-18T13:38:31.116 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:38:30 smithi167 podman[152438]: 2024-06-18 13:38:30.861307957 +0000 UTC m=+0.379540852 container start cfe8643b1bbb6b1e8a06f6c4737251b4da44382ff44bf86669a0d3233e0993b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-3781e3a, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD) 2024-06-18T13:38:31.116 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:38:30 smithi167 podman[152438]: 2024-06-18 13:38:30.941169687 +0000 UTC m=+0.459402573 container attach cfe8643b1bbb6b1e8a06f6c4737251b4da44382ff44bf86669a0d3233e0993b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, RELEASE=squid-3781e3a, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-06-18T13:38:31.410 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 18 13:38:31 smithi167 podman[152438]: 2024-06-18 13:38:31.11687488 +0000 UTC m=+0.635107772 container died cfe8643b1bbb6b1e8a06f6c4737251b4da44382ff44bf86669a0d3233e0993b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-2-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-3781e3a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240612, io.buildah.version=1.36.0) 2024-06-18T13:38:31.687 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.2.service' 2024-06-18T13:38:31.724 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:38:31.725 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-18T13:38:31.725 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-18T13:38:31.725 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.3 2024-06-18T13:38:31.977 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:31 smithi167 systemd[1]: Stopping Ceph osd.3 for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:38:32.477 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:32 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3[58367]: 2024-06-18T13:38:32.072+0000 7f893504f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-18T13:38:32.477 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:32 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3[58367]: 2024-06-18T13:38:32.072+0000 7f893504f640 -1 osd.3 788 *** Got signal Terminated *** 2024-06-18T13:38:32.477 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:32 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3[58367]: 2024-06-18T13:38:32.072+0000 7f893504f640 -1 osd.3 788 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-18T13:38:35.137 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:34 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:38:34.836+0000 7f3abdfa7640 -1 osd.4 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.948682+0000 front 2024-06-18T13:38:09.948571+0000 (oldest deadline 2024-06-18T13:38:34.048568+0000) 2024-06-18T13:38:35.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:35 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:35.008+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:35.988 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:35 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:38:35.822+0000 7f3abdfa7640 -1 osd.4 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.948682+0000 front 2024-06-18T13:38:09.948571+0000 (oldest deadline 2024-06-18T13:38:34.048568+0000) 2024-06-18T13:38:35.989 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:35 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:35.735+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:36.347 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:35 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:35.988+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:36.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:36 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:36.440+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:37.137 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:36 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:36.783+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:37.137 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:36 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:38:36.864+0000 7f3abdfa7640 -1 osd.4 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.948682+0000 front 2024-06-18T13:38:09.948571+0000 (oldest deadline 2024-06-18T13:38:34.048568+0000) 2024-06-18T13:38:37.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:37 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:37.019+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:37.806 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:37 smithi167 podman[152540]: 2024-06-18 13:38:37.553808057 +0000 UTC m=+5.601853767 container died 79de54e0a8dc4693dcf0de001919ed5771a2a12eb790427ac5f235e9ea94c3d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240612, org.label-schema.vendor=CentOS, RELEASE=squid-3781e3a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-18T13:38:37.808 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:37 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:37.408+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:38.098 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:37 smithi167 podman[152540]: 2024-06-18 13:38:37.905505915 +0000 UTC m=+5.953551629 container cleanup 79de54e0a8dc4693dcf0de001919ed5771a2a12eb790427ac5f235e9ea94c3d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3781e3a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240612, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-18T13:38:38.098 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:37 smithi167 bash[152540]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3 2024-06-18T13:38:38.137 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:37 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:38:37.897+0000 7f3abdfa7640 -1 osd.4 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.948682+0000 front 2024-06-18T13:38:09.948571+0000 (oldest deadline 2024-06-18T13:38:34.048568+0000) 2024-06-18T13:38:38.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:37 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:37.997+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:38.137 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:37 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:37.808+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:38.477 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:38 smithi167 podman[152557]: 2024-06-18 13:38:38.098678948 +0000 UTC m=+0.538408844 container remove 79de54e0a8dc4693dcf0de001919ed5771a2a12eb790427ac5f235e9ea94c3d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240612, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3781e3a, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-18T13:38:38.784 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:38 smithi167 podman[152645]: 2024-06-18 13:38:38.516068457 +0000 UTC m=+0.099840226 container create a959a2925c3489b4348a24175b6f83a12a1da870004a16cae449e22ba4a56640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240612, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-3781e3a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-18T13:38:38.785 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:38 smithi167 podman[152645]: 2024-06-18 13:38:38.433437571 +0000 UTC m=+0.017209334 image pull af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314 2024-06-18T13:38:38.785 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:38 smithi167 podman[152645]: 2024-06-18 13:38:38.709902171 +0000 UTC m=+0.293673934 container init a959a2925c3489b4348a24175b6f83a12a1da870004a16cae449e22ba4a56640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-3781e3a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240612, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-18T13:38:38.785 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:38 smithi167 podman[152645]: 2024-06-18 13:38:38.713364736 +0000 UTC m=+0.297136507 container start a959a2925c3489b4348a24175b6f83a12a1da870004a16cae449e22ba4a56640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3781e3a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-18T13:38:38.854 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:38 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:38.455+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:39.137 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:38 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:38.854+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:39.137 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:38 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:38:38.863+0000 7f3abdfa7640 -1 osd.4 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.948682+0000 front 2024-06-18T13:38:09.948571+0000 (oldest deadline 2024-06-18T13:38:34.048568+0000) 2024-06-18T13:38:39.138 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:38 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:38.977+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:39.187 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:38 smithi167 podman[152645]: 2024-06-18 13:38:38.785020628 +0000 UTC m=+0.368792391 container attach a959a2925c3489b4348a24175b6f83a12a1da870004a16cae449e22ba4a56640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3-deactivate, RELEASE=squid-3781e3a, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240612, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-18T13:38:39.187 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:38 smithi167 podman[152645]: 2024-06-18 13:38:38.968291725 +0000 UTC m=+0.552063497 container died a959a2925c3489b4348a24175b6f83a12a1da870004a16cae449e22ba4a56640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3-deactivate, RELEASE=squid-3781e3a, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240612, ceph=True) 2024-06-18T13:38:39.432 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.3.service' 2024-06-18T13:38:39.455 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:39 smithi167 podman[152669]: 2024-06-18 13:38:39.413099576 +0000 UTC m=+0.439281442 container remove a959a2925c3489b4348a24175b6f83a12a1da870004a16cae449e22ba4a56640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3781e3a, ceph=True, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-18T13:38:39.455 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:39 smithi167 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.3.service: Deactivated successfully. 2024-06-18T13:38:39.455 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:39 smithi167 systemd[1]: Stopped Ceph osd.3 for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:38:39.455 INFO:journalctl@ceph.osd.3.smithi167.stdout:Jun 18 13:38:39 smithi167 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.3.service: Consumed 48.934s CPU time. 2024-06-18T13:38:39.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:38:39.470 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-18T13:38:39.470 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-18T13:38:39.470 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.4 2024-06-18T13:38:39.778 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:39 smithi179 systemd[1]: Stopping Ceph osd.4 for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:38:39.779 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:39.491+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:40.137 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:39.897+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:40.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:39.958+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:40.137 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:38:39.777+0000 7f3ac1991640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-18T13:38:40.138 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:38:39.777+0000 7f3ac1991640 -1 osd.4 788 *** Got signal Terminated *** 2024-06-18T13:38:40.138 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:38:39.777+0000 7f3ac1991640 -1 osd.4 788 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-18T13:38:40.138 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:39 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:38:39.913+0000 7f3abdfa7640 -1 osd.4 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.948682+0000 front 2024-06-18T13:38:09.948571+0000 (oldest deadline 2024-06-18T13:38:34.048568+0000) 2024-06-18T13:38:40.873 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:40 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:40.443+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:41.137 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:40 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:38:40.873+0000 7f3abdfa7640 -1 osd.4 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.948682+0000 front 2024-06-18T13:38:09.948571+0000 (oldest deadline 2024-06-18T13:38:34.048568+0000) 2024-06-18T13:38:41.137 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:40 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:40.932+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:41.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:40 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:40.964+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:41.887 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:41 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:41.462+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:42.387 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:41 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:38:41.911+0000 7f3abdfa7640 -1 osd.4 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.948682+0000 front 2024-06-18T13:38:09.948571+0000 (oldest deadline 2024-06-18T13:38:34.048568+0000) 2024-06-18T13:38:42.387 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:41 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:41.948+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:42.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:42 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:42.001+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:42.887 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:42 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:42.420+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:43.384 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:42 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:38:42.919+0000 7f3abdfa7640 -1 osd.4 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.948682+0000 front 2024-06-18T13:38:09.948571+0000 (oldest deadline 2024-06-18T13:38:34.048568+0000) 2024-06-18T13:38:43.384 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:42 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:42.961+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:43.385 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:43 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:43.008+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:43.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:43 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:43.384+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:44.362 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:43 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:43.933+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:44.363 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:43 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:43.968+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:44.363 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:43 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:38:43.921+0000 7f3abdfa7640 -1 osd.4 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.948682+0000 front 2024-06-18T13:38:09.948571+0000 (oldest deadline 2024-06-18T13:38:34.048568+0000) 2024-06-18T13:38:44.363 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:43 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4[38093]: 2024-06-18T13:38:43.921+0000 7f3abdfa7640 -1 osd.4 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:18.149487+0000 front 2024-06-18T13:38:18.149573+0000 (oldest deadline 2024-06-18T13:38:43.449588+0000) 2024-06-18T13:38:44.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:44 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:44.361+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:45.320 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:44 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:44.946+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:45.320 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:44 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:44.946+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:17.546579+0000 front 2024-06-18T13:38:17.546505+0000 (oldest deadline 2024-06-18T13:38:43.946842+0000) 2024-06-18T13:38:45.320 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:44 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:44.946+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:45.603 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:45 smithi179 podman[75214]: 2024-06-18 13:38:45.320210568 +0000 UTC m=+5.646152070 container died 5d5df644e89cc5216ab07c3fd37a56e39fab8b7f2fa3a29c3169be6eb14fcacf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240612, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3781e3a, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-06-18T13:38:45.603 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:45 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:45.350+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:45.887 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:45 smithi179 podman[75214]: 2024-06-18 13:38:45.693529519 +0000 UTC m=+6.019471020 container cleanup 5d5df644e89cc5216ab07c3fd37a56e39fab8b7f2fa3a29c3169be6eb14fcacf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, RELEASE=squid-3781e3a) 2024-06-18T13:38:45.887 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:45 smithi179 bash[75214]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4 2024-06-18T13:38:46.312 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:45 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:45.958+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:46.312 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:45 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:45.958+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:17.546579+0000 front 2024-06-18T13:38:17.546505+0000 (oldest deadline 2024-06-18T13:38:43.946842+0000) 2024-06-18T13:38:46.313 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:45 smithi179 podman[75231]: 2024-06-18 13:38:45.953390678 +0000 UTC m=+0.627976700 container remove 5d5df644e89cc5216ab07c3fd37a56e39fab8b7f2fa3a29c3169be6eb14fcacf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612, RELEASE=squid-3781e3a, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-18T13:38:46.313 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:45 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:45.949+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:46.637 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:46 smithi179 podman[75320]: 2024-06-18 13:38:46.215466503 +0000 UTC m=+0.017686923 image pull af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314 2024-06-18T13:38:46.638 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:46 smithi179 podman[75320]: 2024-06-18 13:38:46.312535716 +0000 UTC m=+0.114756171 container create 7e287b5e59d05a386e23d93c9d85b0dce220cc42dee050fcd70748c8ae5ec3cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3781e3a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-18T13:38:46.638 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:46 smithi179 podman[75320]: 2024-06-18 13:38:46.540043403 +0000 UTC m=+0.342263831 container init 7e287b5e59d05a386e23d93c9d85b0dce220cc42dee050fcd70748c8ae5ec3cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, RELEASE=squid-3781e3a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-18T13:38:46.638 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:46 smithi179 podman[75320]: 2024-06-18 13:38:46.543379874 +0000 UTC m=+0.345600288 container start 7e287b5e59d05a386e23d93c9d85b0dce220cc42dee050fcd70748c8ae5ec3cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240612, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3781e3a, GIT_BRANCH=HEAD) 2024-06-18T13:38:46.638 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:46 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:46.398+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:46.639 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:46 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:46.398+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:38:46.929 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:46 smithi179 podman[75320]: 2024-06-18 13:38:46.639957119 +0000 UTC m=+0.442177548 container attach 7e287b5e59d05a386e23d93c9d85b0dce220cc42dee050fcd70748c8ae5ec3cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-3781e3a) 2024-06-18T13:38:46.929 INFO:journalctl@ceph.osd.4.smithi179.stdout:Jun 18 13:38:46 smithi179 podman[75320]: 2024-06-18 13:38:46.801205868 +0000 UTC m=+0.603426283 container died 7e287b5e59d05a386e23d93c9d85b0dce220cc42dee050fcd70748c8ae5ec3cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-4-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3781e3a, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-18T13:38:47.360 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:46 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:46.929+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:47.360 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:46 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:46.929+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:17.546579+0000 front 2024-06-18T13:38:17.546505+0000 (oldest deadline 2024-06-18T13:38:43.946842+0000) 2024-06-18T13:38:47.360 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:46 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:46.970+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:47.360 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:46 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:46.970+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:38:47.380 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.4.service' 2024-06-18T13:38:47.421 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:38:47.421 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-18T13:38:47.421 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-18T13:38:47.422 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.5 2024-06-18T13:38:47.637 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:47 smithi179 systemd[1]: Stopping Ceph osd.5 for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:38:47.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:47 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:47.402+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:47.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:47 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:47.402+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:38:47.936 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:47 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:47.763+0000 7f6119fb0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-18T13:38:47.936 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:47 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:47.763+0000 7f6119fb0640 -1 osd.5 788 *** Got signal Terminated *** 2024-06-18T13:38:47.937 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:47 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:47.763+0000 7f6119fb0640 -1 osd.5 788 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-18T13:38:47.937 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:47 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:47.935+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:47.937 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:47 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:47.935+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:17.546579+0000 front 2024-06-18T13:38:17.546505+0000 (oldest deadline 2024-06-18T13:38:43.946842+0000) 2024-06-18T13:38:48.361 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:47 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:47.967+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:48.362 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:47 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:47.967+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:38:48.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:48.361+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:48.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:48.361+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:38:49.384 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:48.923+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:49.384 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:48.923+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:17.546579+0000 front 2024-06-18T13:38:17.546505+0000 (oldest deadline 2024-06-18T13:38:43.946842+0000) 2024-06-18T13:38:49.384 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:48.923+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:49.384 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:48 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:48.923+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:38:49.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:49.383+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:49.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:49.383+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:38:50.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:49.917+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:50.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:49.917+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:38:50.137 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:49.885+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:50.137 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:49 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:49.885+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:17.546579+0000 front 2024-06-18T13:38:17.546505+0000 (oldest deadline 2024-06-18T13:38:43.946842+0000) 2024-06-18T13:38:50.880 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:50.431+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:50.880 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:50.431+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:38:51.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:50.881+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:51.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:50.881+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:38:51.137 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:50.880+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:51.137 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:50.880+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:17.546579+0000 front 2024-06-18T13:38:17.546505+0000 (oldest deadline 2024-06-18T13:38:43.946842+0000) 2024-06-18T13:38:51.137 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:50 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:50.880+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:24.447915+0000 front 2024-06-18T13:38:24.447897+0000 (oldest deadline 2024-06-18T13:38:50.347747+0000) 2024-06-18T13:38:51.831 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:51 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:51.427+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:51.831 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:51 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:51.427+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:38:51.831 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:51 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:51.427+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:26.520770+0000 front 2024-06-18T13:38:26.520716+0000 (oldest deadline 2024-06-18T13:38:50.620593+0000) 2024-06-18T13:38:52.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:51 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:51.869+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:52.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:51 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:51.869+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:38:52.137 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:51 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:51.830+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:09.444981+0000 front 2024-06-18T13:38:09.444792+0000 (oldest deadline 2024-06-18T13:38:35.344739+0000) 2024-06-18T13:38:52.137 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:51 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:51.830+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:17.546579+0000 front 2024-06-18T13:38:17.546505+0000 (oldest deadline 2024-06-18T13:38:43.946842+0000) 2024-06-18T13:38:52.138 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:51 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5[43783]: 2024-06-18T13:38:51.830+0000 7f61125be640 -1 osd.5 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:24.447915+0000 front 2024-06-18T13:38:24.447897+0000 (oldest deadline 2024-06-18T13:38:50.347747+0000) 2024-06-18T13:38:52.842 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:52 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:52.462+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:52.842 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:52 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:52.462+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:38:52.842 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:52 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:52.462+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:26.520770+0000 front 2024-06-18T13:38:26.520716+0000 (oldest deadline 2024-06-18T13:38:50.620593+0000) 2024-06-18T13:38:53.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:52 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:52.841+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:53.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:52 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:52.841+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:38:53.447 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:53 smithi179 podman[75424]: 2024-06-18 13:38:53.152919924 +0000 UTC m=+5.480185346 container died 8ea9510e0f4d103619129eed99ffc44e459d098fb6694315360695d61a3f66b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-3781e3a, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, org.label-schema.license=GPLv2) 2024-06-18T13:38:53.798 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:53 smithi179 podman[75424]: 2024-06-18 13:38:53.53859979 +0000 UTC m=+5.865865215 container cleanup 8ea9510e0f4d103619129eed99ffc44e459d098fb6694315360695d61a3f66b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240612, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3781e3a, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-18T13:38:53.798 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:53 smithi179 bash[75424]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5 2024-06-18T13:38:53.799 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:53 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:53.494+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:53.799 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:53 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:53.494+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:38:53.799 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:53 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:53.494+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:26.520770+0000 front 2024-06-18T13:38:26.520716+0000 (oldest deadline 2024-06-18T13:38:50.620593+0000) 2024-06-18T13:38:54.115 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:53 smithi179 podman[75453]: 2024-06-18 13:38:53.798599648 +0000 UTC m=+0.636198892 container remove 8ea9510e0f4d103619129eed99ffc44e459d098fb6694315360695d61a3f66b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3781e3a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612) 2024-06-18T13:38:54.115 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:53 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:53.797+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:54.116 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:53 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:53.797+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:38:54.387 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:54 smithi179 podman[75540]: 2024-06-18 13:38:54.116029345 +0000 UTC m=+0.112680885 container create dc0244b7bde0659807687074e4f05e4b52e0c516550173137c40f5cb0ecd8484 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240612, ceph=True, RELEASE=squid-3781e3a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-06-18T13:38:54.387 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:54 smithi179 podman[75540]: 2024-06-18 13:38:54.020589173 +0000 UTC m=+0.017240726 image pull af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314 2024-06-18T13:38:54.700 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:54 smithi179 podman[75540]: 2024-06-18 13:38:54.435427781 +0000 UTC m=+0.432079327 container init dc0244b7bde0659807687074e4f05e4b52e0c516550173137c40f5cb0ecd8484 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-3781e3a, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-18T13:38:54.700 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:54 smithi179 podman[75540]: 2024-06-18 13:38:54.438808117 +0000 UTC m=+0.435459658 container start dc0244b7bde0659807687074e4f05e4b52e0c516550173137c40f5cb0ecd8484 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3781e3a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-06-18T13:38:54.701 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:54 smithi179 podman[75540]: 2024-06-18 13:38:54.518504149 +0000 UTC m=+0.515155717 container attach dc0244b7bde0659807687074e4f05e4b52e0c516550173137c40f5cb0ecd8484 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3781e3a, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.build-date=20240612) 2024-06-18T13:38:54.701 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:54 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:54.467+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:54.701 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:54 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:54.467+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:38:54.701 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:54 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:54.467+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:26.520770+0000 front 2024-06-18T13:38:26.520716+0000 (oldest deadline 2024-06-18T13:38:50.620593+0000) 2024-06-18T13:38:55.004 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:54 smithi179 conmon[75555]: conmon dc0244b7bde065980768 : Failed to open cgroups file: /sys/fs/cgroup/machine.slice/libpod-dc0244b7bde0659807687074e4f05e4b52e0c516550173137c40f5cb0ecd8484.scope/container/memory.events 2024-06-18T13:38:55.005 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 18 13:38:54 smithi179 podman[75540]: 2024-06-18 13:38:54.701596418 +0000 UTC m=+0.698247960 container died dc0244b7bde0659807687074e4f05e4b52e0c516550173137c40f5cb0ecd8484 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-5-deactivate, RELEASE=squid-3781e3a, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-18T13:38:55.005 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:54 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:54.782+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:55.005 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:54 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:54.782+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:38:55.005 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:54 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:54.782+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:29.308863+0000 front 2024-06-18T13:38:29.308804+0000 (oldest deadline 2024-06-18T13:38:54.608760+0000) 2024-06-18T13:38:55.307 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.5.service' 2024-06-18T13:38:55.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:38:55.346 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-18T13:38:55.346 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-18T13:38:55.347 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.6 2024-06-18T13:38:55.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:55 smithi179 systemd[1]: Stopping Ceph osd.6 for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:38:55.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:55 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:55.493+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:55.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:55 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:55.493+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:38:55.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:55 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:55.493+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:26.520770+0000 front 2024-06-18T13:38:26.520716+0000 (oldest deadline 2024-06-18T13:38:50.620593+0000) 2024-06-18T13:38:56.137 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:55 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:55.674+0000 7f273044d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-18T13:38:56.137 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:55 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:55.674+0000 7f273044d640 -1 osd.6 788 *** Got signal Terminated *** 2024-06-18T13:38:56.137 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:55 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:55.674+0000 7f273044d640 -1 osd.6 788 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-18T13:38:56.138 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:55 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:55.808+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:56.138 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:55 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:55.808+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:38:56.138 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:55 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:55.808+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:29.308863+0000 front 2024-06-18T13:38:29.308804+0000 (oldest deadline 2024-06-18T13:38:54.608760+0000) 2024-06-18T13:38:56.841 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:56 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:56.538+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:56.841 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:56 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:56.538+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:38:56.841 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:56 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:56.538+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:26.520770+0000 front 2024-06-18T13:38:26.520716+0000 (oldest deadline 2024-06-18T13:38:50.620593+0000) 2024-06-18T13:38:57.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:56 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:56.839+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:57.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:56 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:56.839+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:38:57.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:56 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:56.839+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:29.308863+0000 front 2024-06-18T13:38:29.308804+0000 (oldest deadline 2024-06-18T13:38:54.608760+0000) 2024-06-18T13:38:57.887 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:57 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:57.552+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:57.887 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:57 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:57.552+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:38:57.887 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:57 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:57.552+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:26.520770+0000 front 2024-06-18T13:38:26.520716+0000 (oldest deadline 2024-06-18T13:38:50.620593+0000) 2024-06-18T13:38:58.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:57 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:57.889+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:58.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:57 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:57.889+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:38:58.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:57 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:57.889+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:29.308863+0000 front 2024-06-18T13:38:29.308804+0000 (oldest deadline 2024-06-18T13:38:54.608760+0000) 2024-06-18T13:38:58.877 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:58.540+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:58.877 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:58.540+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:38:58.877 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:58.540+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:26.520770+0000 front 2024-06-18T13:38:26.520716+0000 (oldest deadline 2024-06-18T13:38:50.620593+0000) 2024-06-18T13:38:59.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:58.877+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:38:59.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:58.877+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:38:59.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:58 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:58.877+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:29.308863+0000 front 2024-06-18T13:38:29.308804+0000 (oldest deadline 2024-06-18T13:38:54.608760+0000) 2024-06-18T13:38:59.879 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:59.539+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:38:59.879 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:59.539+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:38:59.879 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:59.539+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:26.520770+0000 front 2024-06-18T13:38:26.520716+0000 (oldest deadline 2024-06-18T13:38:50.620593+0000) 2024-06-18T13:38:59.879 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:38:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:38:59.539+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6830 osd.3 since back 2024-06-18T13:38:35.321478+0000 front 2024-06-18T13:38:35.321492+0000 (oldest deadline 2024-06-18T13:38:58.821438+0000) 2024-06-18T13:39:00.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:59.878+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:39:00.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:59.878+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:39:00.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:38:59 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:38:59.878+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:29.308863+0000 front 2024-06-18T13:38:29.308804+0000 (oldest deadline 2024-06-18T13:38:54.608760+0000) 2024-06-18T13:39:00.887 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:39:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:39:00.555+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:12.618634+0000 front 2024-06-18T13:38:12.618622+0000 (oldest deadline 2024-06-18T13:38:36.118399+0000) 2024-06-18T13:39:00.887 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:39:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:39:00.555+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.719921+0000 front 2024-06-18T13:38:20.719911+0000 (oldest deadline 2024-06-18T13:38:45.419792+0000) 2024-06-18T13:39:00.887 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:39:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:39:00.555+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:26.520770+0000 front 2024-06-18T13:38:26.520716+0000 (oldest deadline 2024-06-18T13:38:50.620593+0000) 2024-06-18T13:39:00.887 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:39:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6[49534]: 2024-06-18T13:39:00.555+0000 7f2728a5b640 -1 osd.6 788 heartbeat_check: no reply from 172.21.15.167:6830 osd.3 since back 2024-06-18T13:38:35.321478+0000 front 2024-06-18T13:38:35.321492+0000 (oldest deadline 2024-06-18T13:38:58.821438+0000) 2024-06-18T13:39:01.310 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:39:01 smithi179 podman[75643]: 2024-06-18 13:39:01.055190342 +0000 UTC m=+5.482615325 container died 07d1a826acabed233305bba0eb37ea159a9d8095d32c2ee3261757eccd7ff44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240612, RELEASE=squid-3781e3a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True) 2024-06-18T13:39:01.310 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:00.925+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:39:01.310 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:00.925+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:39:01.310 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:00.925+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:29.308863+0000 front 2024-06-18T13:38:29.308804+0000 (oldest deadline 2024-06-18T13:38:54.608760+0000) 2024-06-18T13:39:01.310 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:00 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:00.925+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6830 osd.3 since back 2024-06-18T13:38:34.609296+0000 front 2024-06-18T13:38:34.609324+0000 (oldest deadline 2024-06-18T13:39:00.509185+0000) 2024-06-18T13:39:01.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:39:01 smithi179 podman[75643]: 2024-06-18 13:39:01.408173348 +0000 UTC m=+5.835598330 container cleanup 07d1a826acabed233305bba0eb37ea159a9d8095d32c2ee3261757eccd7ff44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6, RELEASE=squid-3781e3a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, ceph=True) 2024-06-18T13:39:01.637 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:39:01 smithi179 bash[75643]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6 2024-06-18T13:39:01.958 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:39:01 smithi179 podman[75658]: 2024-06-18 13:39:01.66774636 +0000 UTC m=+0.612088223 container remove 07d1a826acabed233305bba0eb37ea159a9d8095d32c2ee3261757eccd7ff44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=squid-3781e3a, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-18T13:39:02.270 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:01 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:01.957+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:39:02.271 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:01 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:01.957+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:39:02.271 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:01 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:01.957+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:29.308863+0000 front 2024-06-18T13:38:29.308804+0000 (oldest deadline 2024-06-18T13:38:54.608760+0000) 2024-06-18T13:39:02.271 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:01 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:01.957+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6830 osd.3 since back 2024-06-18T13:38:34.609296+0000 front 2024-06-18T13:38:34.609324+0000 (oldest deadline 2024-06-18T13:39:00.509185+0000) 2024-06-18T13:39:02.271 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:39:02 smithi179 podman[75747]: 2024-06-18 13:39:01.935201659 +0000 UTC m=+0.017094745 image pull af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314 2024-06-18T13:39:02.271 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:39:02 smithi179 podman[75747]: 2024-06-18 13:39:02.06029302 +0000 UTC m=+0.142186141 container create 963fdbe0ecf35ff6ccb4fa3449cdb78c548426d1764083f9e36acdbc831da527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6-deactivate, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-3781e3a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-06-18T13:39:02.271 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:39:02 smithi179 podman[75747]: 2024-06-18 13:39:02.271117884 +0000 UTC m=+0.353010970 container init 963fdbe0ecf35ff6ccb4fa3449cdb78c548426d1764083f9e36acdbc831da527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240612, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3781e3a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-18T13:39:02.542 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:39:02 smithi179 podman[75747]: 2024-06-18 13:39:02.274493242 +0000 UTC m=+0.356386318 container start 963fdbe0ecf35ff6ccb4fa3449cdb78c548426d1764083f9e36acdbc831da527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240612, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3781e3a, org.label-schema.license=GPLv2) 2024-06-18T13:39:02.542 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:39:02 smithi179 podman[75747]: 2024-06-18 13:39:02.35419431 +0000 UTC m=+0.436087395 container attach 963fdbe0ecf35ff6ccb4fa3449cdb78c548426d1764083f9e36acdbc831da527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3781e3a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-18T13:39:02.849 INFO:journalctl@ceph.osd.6.smithi179.stdout:Jun 18 13:39:02 smithi179 podman[75747]: 2024-06-18 13:39:02.542574904 +0000 UTC m=+0.624467988 container died 963fdbe0ecf35ff6ccb4fa3449cdb78c548426d1764083f9e36acdbc831da527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-6-deactivate, RELEASE=squid-3781e3a, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-06-18T13:39:03.107 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:02 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:02.964+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:39:03.107 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:02 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:02.964+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:39:03.107 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:02 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:02.964+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:29.308863+0000 front 2024-06-18T13:38:29.308804+0000 (oldest deadline 2024-06-18T13:38:54.608760+0000) 2024-06-18T13:39:03.107 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:02 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:02.964+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6830 osd.3 since back 2024-06-18T13:38:34.609296+0000 front 2024-06-18T13:38:34.609324+0000 (oldest deadline 2024-06-18T13:39:00.509185+0000) 2024-06-18T13:39:03.128 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.6.service' 2024-06-18T13:39:03.166 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:39:03.166 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-18T13:39:03.166 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-18T13:39:03.166 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.7 2024-06-18T13:39:03.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:03 smithi179 systemd[1]: Stopping Ceph osd.7 for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:39:03.887 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:03 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:03.485+0000 7f600e07d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-18T13:39:03.887 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:03 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:03.485+0000 7f600e07d640 -1 osd.7 788 *** Got signal Terminated *** 2024-06-18T13:39:03.887 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:03 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:03.485+0000 7f600e07d640 -1 osd.7 788 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-18T13:39:04.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:03 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:03.998+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:39:04.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:03 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:03.998+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:39:04.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:03 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:03.998+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:29.308863+0000 front 2024-06-18T13:38:29.308804+0000 (oldest deadline 2024-06-18T13:38:54.608760+0000) 2024-06-18T13:39:04.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:03 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:03.998+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6830 osd.3 since back 2024-06-18T13:38:34.609296+0000 front 2024-06-18T13:38:34.609324+0000 (oldest deadline 2024-06-18T13:39:00.509185+0000) 2024-06-18T13:39:05.289 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:04 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:04.960+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:39:05.289 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:04 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:04.960+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:39:05.289 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:04 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:04.960+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:29.308863+0000 front 2024-06-18T13:38:29.308804+0000 (oldest deadline 2024-06-18T13:38:54.608760+0000) 2024-06-18T13:39:05.289 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:04 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:04.960+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6830 osd.3 since back 2024-06-18T13:38:34.609296+0000 front 2024-06-18T13:38:34.609324+0000 (oldest deadline 2024-06-18T13:39:00.509185+0000) 2024-06-18T13:39:06.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:05 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:05.923+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:39:06.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:05 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:05.923+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:39:06.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:05 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:05.923+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:29.308863+0000 front 2024-06-18T13:38:29.308804+0000 (oldest deadline 2024-06-18T13:38:54.608760+0000) 2024-06-18T13:39:06.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:05 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:05.923+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6830 osd.3 since back 2024-06-18T13:38:34.609296+0000 front 2024-06-18T13:38:34.609324+0000 (oldest deadline 2024-06-18T13:39:00.509185+0000) 2024-06-18T13:39:07.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:06 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:06.941+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:39:07.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:06 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:06.941+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:39:07.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:06 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:06.941+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:29.308863+0000 front 2024-06-18T13:38:29.308804+0000 (oldest deadline 2024-06-18T13:38:54.608760+0000) 2024-06-18T13:39:07.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:06 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:06.941+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6830 osd.3 since back 2024-06-18T13:38:34.609296+0000 front 2024-06-18T13:38:34.609324+0000 (oldest deadline 2024-06-18T13:39:00.509185+0000) 2024-06-18T13:39:07.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:06 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:06.941+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.179:6804 osd.4 since back 2024-06-18T13:38:40.509804+0000 front 2024-06-18T13:38:40.509571+0000 (oldest deadline 2024-06-18T13:39:06.409557+0000) 2024-06-18T13:39:08.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:07 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:07.924+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6806 osd.0 since back 2024-06-18T13:38:10.706294+0000 front 2024-06-18T13:38:10.706172+0000 (oldest deadline 2024-06-18T13:38:34.805966+0000) 2024-06-18T13:39:08.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:07 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:07.924+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6814 osd.1 since back 2024-06-18T13:38:20.706873+0000 front 2024-06-18T13:38:20.707000+0000 (oldest deadline 2024-06-18T13:38:46.606817+0000) 2024-06-18T13:39:08.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:07 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:07.924+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6822 osd.2 since back 2024-06-18T13:38:29.308863+0000 front 2024-06-18T13:38:29.308804+0000 (oldest deadline 2024-06-18T13:38:54.608760+0000) 2024-06-18T13:39:08.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:07 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:07.924+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.167:6830 osd.3 since back 2024-06-18T13:38:34.609296+0000 front 2024-06-18T13:38:34.609324+0000 (oldest deadline 2024-06-18T13:39:00.509185+0000) 2024-06-18T13:39:08.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:07 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7[55068]: 2024-06-18T13:39:07.924+0000 7f600668b640 -1 osd.7 788 heartbeat_check: no reply from 172.21.15.179:6804 osd.4 since back 2024-06-18T13:38:40.509804+0000 front 2024-06-18T13:38:40.509571+0000 (oldest deadline 2024-06-18T13:39:06.409557+0000) 2024-06-18T13:39:09.135 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:08 smithi179 podman[75848]: 2024-06-18 13:39:08.835524204 +0000 UTC m=+5.446366394 container died 3fc0fafb8f8a0ad424dc7a56986997ef22a0b0699d76a8dd40fe3d71687b896c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240612, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3781e3a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-18T13:39:09.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:09 smithi179 podman[75848]: 2024-06-18 13:39:09.218358867 +0000 UTC m=+5.829201098 container cleanup 3fc0fafb8f8a0ad424dc7a56986997ef22a0b0699d76a8dd40fe3d71687b896c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3781e3a, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240612, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-18T13:39:09.387 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:09 smithi179 bash[75848]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7 2024-06-18T13:39:09.770 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:09 smithi179 podman[75878]: 2024-06-18 13:39:09.436157458 +0000 UTC m=+0.603430900 container remove 3fc0fafb8f8a0ad424dc7a56986997ef22a0b0699d76a8dd40fe3d71687b896c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3781e3a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-18T13:39:10.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:09 smithi179 podman[75965]: 2024-06-18 13:39:09.770231782 +0000 UTC m=+0.110565585 container create ad33cb05826958c0415e7a2ec817118e3418ced751810f9669eebdb4e5370808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3781e3a, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-18T13:39:10.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:09 smithi179 podman[75965]: 2024-06-18 13:39:09.677404147 +0000 UTC m=+0.017737909 image pull af93c036e0161a7a6f4fcb3f1f48acaac5cf4ef8953c6a8b104629a4b86fec44 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314 2024-06-18T13:39:10.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:09 smithi179 podman[75965]: 2024-06-18 13:39:09.947751641 +0000 UTC m=+0.288085402 container init ad33cb05826958c0415e7a2ec817118e3418ced751810f9669eebdb4e5370808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, RELEASE=squid-3781e3a, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-18T13:39:10.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:09 smithi179 podman[75965]: 2024-06-18 13:39:09.951072222 +0000 UTC m=+0.291405992 container start ad33cb05826958c0415e7a2ec817118e3418ced751810f9669eebdb4e5370808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7-deactivate, RELEASE=squid-3781e3a, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-06-18T13:39:10.137 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:10 smithi179 podman[75965]: 2024-06-18 13:39:10.014240593 +0000 UTC m=+0.354574356 container attach ad33cb05826958c0415e7a2ec817118e3418ced751810f9669eebdb4e5370808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3781e3a) 2024-06-18T13:39:10.500 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:10 smithi179 podman[75965]: 2024-06-18 13:39:10.211002686 +0000 UTC m=+0.551336446 container died ad33cb05826958c0415e7a2ec817118e3418ced751810f9669eebdb4e5370808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3781e3a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240612, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-18T13:39:10.764 INFO:journalctl@ceph.osd.7.smithi179.stdout:Jun 18 13:39:10 smithi179 podman[75987]: 2024-06-18 13:39:10.74213852 +0000 UTC m=+0.525131252 container remove ad33cb05826958c0415e7a2ec817118e3418ced751810f9669eebdb4e5370808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-osd-7-deactivate, RELEASE=squid-3781e3a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0) 2024-06-18T13:39:10.770 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@osd.7.service' 2024-06-18T13:39:10.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:39:10.810 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-18T13:39:10.810 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-06-18T13:39:10.810 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@rgw.foo.a 2024-06-18T13:39:11.105 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:39:10 smithi167 systemd[1]: Stopping Ceph rgw.foo.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:39:11.362 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:39:11 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-rgw-foo-a[63629]: 2024-06-18T13:39:11.104+0000 7f29da47b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-18T13:39:11.362 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:39:11 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-rgw-foo-a[63629]: 2024-06-18T13:39:11.104+0000 7f29ddce7880 -1 shutting down 2024-06-18T13:39:21.388 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:39:21 smithi167 bash[152750]: time="2024-06-18T13:39:21Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-06-18T13:39:21.388 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:39:21 smithi167 podman[152750]: 2024-06-18 13:39:21.125897766 +0000 UTC m=+10.118252422 container died 726db2c5d89f783ca7488a694be2443acc6c76107b42e396e9b5f0c5c84a5eea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-rgw-foo-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3781e3a, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-06-18T13:39:21.727 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:39:21 smithi167 podman[152750]: 2024-06-18 13:39:21.484579899 +0000 UTC m=+10.476934552 container cleanup 726db2c5d89f783ca7488a694be2443acc6c76107b42e396e9b5f0c5c84a5eea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-rgw-foo-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-3781e3a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240612, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-06-18T13:39:21.727 INFO:journalctl@ceph.rgw.foo.a.smithi167.stdout:Jun 18 13:39:21 smithi167 bash[152750]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-rgw-foo-a 2024-06-18T13:39:21.872 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@rgw.foo.a.service' 2024-06-18T13:39:21.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:39:21.909 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-06-18T13:39:21.909 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-06-18T13:39:21.909 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@prometheus.a 2024-06-18T13:39:22.202 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:21 smithi179 systemd[1]: Stopping Ceph prometheus.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:39:22.470 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:39:22.202Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-06-18T13:39:22.470 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:39:22.202Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-06-18T13:39:22.470 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:39:22.202Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-06-18T13:39:22.470 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:39:22.202Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-06-18T13:39:22.470 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:39:22.202Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-06-18T13:39:22.470 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:39:22.202Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-06-18T13:39:22.470 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:39:22.202Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-06-18T13:39:22.471 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:39:22.202Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-06-18T13:39:22.471 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:39:22.203Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-06-18T13:39:22.471 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:39:22.204Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-06-18T13:39:22.471 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:39:22.204Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-06-18T13:39:22.471 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a[64656]: ts=2024-06-18T13:39:22.204Z caller=main.go:1147 level=info msg="See you next time!" 2024-06-18T13:39:22.471 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 podman[76066]: 2024-06-18 13:39:22.212995723 +0000 UTC m=+0.105980129 container died 4d7013e6be9e26f8d87e0546fed72dad06f16b2176aa3971271593cb92bd05b0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-18T13:39:22.836 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 podman[76066]: 2024-06-18 13:39:22.577281323 +0000 UTC m=+0.470265727 container cleanup 4d7013e6be9e26f8d87e0546fed72dad06f16b2176aa3971271593cb92bd05b0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-18T13:39:22.836 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Jun 18 13:39:22 smithi179 bash[76066]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-prometheus-a 2024-06-18T13:39:23.014 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@prometheus.a.service' 2024-06-18T13:39:23.048 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-18T13:39:23.048 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-06-18T13:39:23.049 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 --force --keep-logs 2024-06-18T13:39:23.239 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T13:39:24.977 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:39:24 smithi167 systemd[1]: Stopping Ceph alertmanager.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:39:25.368 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:39:25 smithi167 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a[73175]: ts=2024-06-18T13:39:25.067Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-06-18T13:39:25.368 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:39:25 smithi167 podman[153255]: 2024-06-18 13:39:25.108477202 +0000 UTC m=+0.142812963 container died 60a8f2c52c41af9d152f7aea0b6000d6e8d6db741793aab0d992e015bbbbef52 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-18T13:39:25.718 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:39:25 smithi167 podman[153255]: 2024-06-18 13:39:25.459506364 +0000 UTC m=+0.493842163 container cleanup 60a8f2c52c41af9d152f7aea0b6000d6e8d6db741793aab0d992e015bbbbef52 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-18T13:39:25.718 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:39:25 smithi167 bash[153255]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a 2024-06-18T13:39:25.977 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:39:25 smithi167 podman[153268]: 2024-06-18 13:39:25.719125486 +0000 UTC m=+0.604929253 container remove 60a8f2c52c41af9d152f7aea0b6000d6e8d6db741793aab0d992e015bbbbef52 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-18T13:39:25.977 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:39:25 smithi167 podman[153268]: 2024-06-18 13:39:25.777940599 +0000 UTC m=+0.663744369 volume remove 08e6397bddcbbbbb8301af78c36d11f3f7016e1b7380682f685beb94b9211c7e 2024-06-18T13:39:25.977 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:39:25 smithi167 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-06-18T13:39:25.977 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:39:25 smithi167 systemd[1]: Stopped Ceph alertmanager.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:39:25.977 INFO:journalctl@ceph.alertmanager.a.smithi167.stdout:Jun 18 13:39:25 smithi167 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@alertmanager.a.service: Consumed 2.381s CPU time. 2024-06-18T13:39:26.477 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:39:26 smithi167 systemd[1]: Stopping Ceph node-exporter.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:39:26.477 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:39:26 smithi167 podman[153381]: 2024-06-18 13:39:26.366396795 +0000 UTC m=+0.102316378 container died 10ac283a363106c49ac0662bce4715e2d296e5d5658f2a3bdf3f30cb34400d10 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-18T13:39:26.881 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:39:26 smithi167 podman[153381]: 2024-06-18 13:39:26.705724824 +0000 UTC m=+0.441644408 container cleanup 10ac283a363106c49ac0662bce4715e2d296e5d5658f2a3bdf3f30cb34400d10 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-18T13:39:26.881 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:39:26 smithi167 bash[153381]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a 2024-06-18T13:39:27.217 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:39:26 smithi167 podman[153393]: 2024-06-18 13:39:26.881814722 +0000 UTC m=+0.519304041 container remove 10ac283a363106c49ac0662bce4715e2d296e5d5658f2a3bdf3f30cb34400d10 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-18T13:39:27.217 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:39:26 smithi167 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-06-18T13:39:27.217 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:39:27 smithi167 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-06-18T13:39:27.217 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:39:27 smithi167 systemd[1]: Stopped Ceph node-exporter.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:39:27.217 INFO:journalctl@ceph.node-exporter.a.smithi167.stdout:Jun 18 13:39:27 smithi167 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@node-exporter.a.service: Consumed 4.976s CPU time. 2024-06-18T13:39:27.853 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 --force --keep-logs 2024-06-18T13:39:28.046 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T13:39:29.387 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:39:29 smithi179 systemd[1]: Stopping Ceph iscsi.iscsi.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:39:29.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:39:29 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a[59780]: debug Shutdown received 2024-06-18T13:39:39.791 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:39:39 smithi179 bash[76475]: time="2024-06-18T13:39:39Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-06-18T13:39:39.791 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:39:39 smithi179 podman[76475]: 2024-06-18 13:39:39.524944991 +0000 UTC m=+10.115162807 container died dd8566daf6c4f52f5c51665e5561a6bdcedd6170c052470814fa07809d8cfc55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3781e3a, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240612, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-06-18T13:39:40.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:39:39 smithi179 podman[76475]: 2024-06-18 13:39:39.855789269 +0000 UTC m=+10.446007085 container cleanup dd8566daf6c4f52f5c51665e5561a6bdcedd6170c052470814fa07809d8cfc55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-3781e3a, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-06-18T13:39:40.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:39:39 smithi179 bash[76475]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a 2024-06-18T13:39:40.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:39:40 smithi179 podman[76488]: 2024-06-18 13:39:40.032050798 +0000 UTC m=+0.504049579 container remove dd8566daf6c4f52f5c51665e5561a6bdcedd6170c052470814fa07809d8cfc55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:64b08d1c422bf6aa753839cabed8a69924a54ceb6ef0660c2e561d19ca874314, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-iscsi-iscsi-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-3781e3a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-18T13:39:40.137 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:39:40 smithi179 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-06-18T13:39:40.603 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:39:40 smithi179 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-06-18T13:39:40.604 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:39:40 smithi179 systemd[1]: Stopped Ceph iscsi.iscsi.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:39:40.604 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Jun 18 13:39:40 smithi179 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@iscsi.iscsi.a.service: Consumed 4.455s CPU time. 2024-06-18T13:39:42.081 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:39:41 smithi179 systemd[1]: Stopping Ceph grafana.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:39:42.387 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:39:42 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=server t=2024-06-18T13:39:42.081494903Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-06-18T13:39:42.387 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:39:42 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=ticker t=2024-06-18T13:39:42.081793858Z level=info msg=stopped last_tick=2024-06-18T13:39:40Z 2024-06-18T13:39:42.387 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:39:42 smithi179 ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a[63337]: logger=tracing t=2024-06-18T13:39:42.081781168Z level=info msg="Closing tracing" 2024-06-18T13:39:42.387 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:39:42 smithi179 podman[76761]: 2024-06-18 13:39:42.091796353 +0000 UTC m=+0.126178541 container died 87355cb584c4ad47ef1655280d05a4b419e488b37344e98765910f3717e9a0d4 (image=quay.io/ceph/grafana:9.4.12, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-06-18T13:39:42.656 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:39:42 smithi179 podman[76761]: 2024-06-18 13:39:42.405509179 +0000 UTC m=+0.439891366 container cleanup 87355cb584c4ad47ef1655280d05a4b419e488b37344e98765910f3717e9a0d4 (image=quay.io/ceph/grafana:9.4.12, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-06-18T13:39:42.656 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:39:42 smithi179 bash[76761]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a 2024-06-18T13:39:42.656 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:39:42 smithi179 podman[76773]: 2024-06-18 13:39:42.590154038 +0000 UTC m=+0.495699690 container remove 87355cb584c4ad47ef1655280d05a4b419e488b37344e98765910f3717e9a0d4 (image=quay.io/ceph/grafana:9.4.12, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-06-18T13:39:43.021 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:39:42 smithi179 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@grafana.a.service: Deactivated successfully. 2024-06-18T13:39:43.021 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:39:42 smithi179 systemd[1]: Stopped Ceph grafana.a for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:39:43.021 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Jun 18 13:39:42 smithi179 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@grafana.a.service: Consumed 22.198s CPU time. 2024-06-18T13:39:43.287 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:39:43 smithi179 systemd[1]: Stopping Ceph node-exporter.b for 3434f1e2-2d72-11ef-bc9d-c7b262605968... 2024-06-18T13:39:43.561 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:39:43 smithi179 podman[76890]: 2024-06-18 13:39:43.287801355 +0000 UTC m=+0.127819373 container died 8248ccef56c1e12bd30e0caaf657a71bd35216a8d20a2128a25d8d3b5035e891 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-18T13:39:43.887 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:39:43 smithi179 podman[76890]: 2024-06-18 13:39:43.668526523 +0000 UTC m=+0.508544533 container cleanup 8248ccef56c1e12bd30e0caaf657a71bd35216a8d20a2128a25d8d3b5035e891 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-18T13:39:43.887 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:39:43 smithi179 bash[76890]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b 2024-06-18T13:39:44.188 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:39:43 smithi179 podman[76902]: 2024-06-18 13:39:43.903054244 +0000 UTC m=+0.615288338 container remove 8248ccef56c1e12bd30e0caaf657a71bd35216a8d20a2128a25d8d3b5035e891 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-18T13:39:44.189 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:39:43 smithi179 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-06-18T13:39:44.189 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:39:44 smithi179 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-06-18T13:39:44.189 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:39:44 smithi179 systemd[1]: Stopped Ceph node-exporter.b for 3434f1e2-2d72-11ef-bc9d-c7b262605968. 2024-06-18T13:39:44.189 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Jun 18 13:39:44 smithi179 systemd[1]: ceph-3434f1e2-2d72-11ef-bc9d-c7b262605968@node-exporter.b.service: Consumed 5.101s CPU time. 2024-06-18T13:39:44.696 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-18T13:39:44.729 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-18T13:39:44.757 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-18T13:39:44.758 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-17_21:08:03-orch-squid-distro-default-smithi/7760537/remote/smithi167/crash 2024-06-18T13:39:44.759 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/crash -- . 2024-06-18T13:39:44.860 INFO:teuthology.orchestra.run.smithi167.stderr:tar: /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/crash: Cannot open: No such file or directory 2024-06-18T13:39:44.860 INFO:teuthology.orchestra.run.smithi167.stderr:tar: Error is not recoverable: exiting now 2024-06-18T13:39:44.862 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-17_21:08:03-orch-squid-distro-default-smithi/7760537/remote/smithi179/crash 2024-06-18T13:39:44.863 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/crash -- . 2024-06-18T13:39:44.900 INFO:teuthology.orchestra.run.smithi179.stderr:tar: /var/lib/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/crash: Cannot open: No such file or directory 2024-06-18T13:39:44.900 INFO:teuthology.orchestra.run.smithi179.stderr:tar: Error is not recoverable: exiting now 2024-06-18T13:39:44.902 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-18T13:39:44.902 DEBUG:teuthology.orchestra.run.smithi167:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-06-18T13:39:44.949 INFO:teuthology.orchestra.run.smithi167.stdout:2024-06-18T13:05:38.388638+0000 mon.a (mon.0) 1238 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-18T13:39:44.949 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-18T13:39:44.950 DEBUG:teuthology.orchestra.run.smithi167:> sudo egrep '\[SEC\]' /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-06-18T13:39:44.980 DEBUG:teuthology.orchestra.run.smithi167:> sudo egrep '\[ERR\]' /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-06-18T13:39:45.052 DEBUG:teuthology.orchestra.run.smithi167:> sudo egrep '\[WRN\]' /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-06-18T13:39:45.120 INFO:teuthology.orchestra.run.smithi167.stdout:2024-06-18T13:05:38.388638+0000 mon.a (mon.0) 1238 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-18T13:39:45.121 INFO:tasks.cephadm:Compressing logs... 2024-06-18T13:39:45.121 DEBUG:teuthology.orchestra.run.smithi167:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-18T13:39:45.163 DEBUG:teuthology.orchestra.run.smithi179:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-18T13:39:45.184 INFO:teuthology.orchestra.run.smithi167.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-06-18T13:39:45.185 INFO:teuthology.orchestra.run.smithi167.stderr: /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.log 2024-06-18T13:39:45.185 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.2.log 2024-06-18T13:39:45.187 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.audit.log 2024-06-18T13:39:45.188 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.3.log 2024-06-18T13:39:45.188 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.1.log 2024-06-18T13:39:45.188 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mgr.y.log 2024-06-18T13:39:45.188 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.0.log 2024-06-18T13:39:45.188 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mon.c.log 2024-06-18T13:39:45.189 INFO:teuthology.orchestra.run.smithi179.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.log 2024-06-18T13:39:45.189 INFO:teuthology.orchestra.run.smithi179.stderr:: No such file or directory 2024-06-18T13:39:45.189 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.audit.log 2024-06-18T13:39:45.189 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.7.log 2024-06-18T13:39:45.189 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.log: gzip/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.audit.log: -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.5.log 2024-06-18T13:39:45.189 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.4.log 2024-06-18T13:39:45.192 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mgr.x.log 2024-06-18T13:39:45.193 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.6.log 2024-06-18T13:39:45.193 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.cephadm.log 2024-06-18T13:39:45.193 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.6.log: /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.cephadm.log: 79.8% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.cephadm.log.gz 2024-06-18T13:39:45.193 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/tcmu-runner.log 2024-06-18T13:39:45.193 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-volume.log 2024-06-18T13:39:45.193 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/tcmu-runner.log: 62.9% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/tcmu-runner.log.gz 2024-06-18T13:39:45.193 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mon.b.log 2024-06-18T13:39:45.194 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-volume.log: 88.1% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.log.gz 2024-06-18T13:39:45.194 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-18T13:39:45.195 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.0.log: 93.4% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.log.gz 2024-06-18T13:39:45.196 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mon.a.log 2024-06-18T13:39:45.199 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mon.b.log: 92.5% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.audit.log.gz 2024-06-18T13:39:45.203 INFO:teuthology.orchestra.run.smithi179.stderr: 92.7% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mgr.x.log.gz 2024-06-18T13:39:45.205 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mon.c.log: 95.4% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.audit.log.gz 2024-06-18T13:39:45.206 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.cephadm.log 2024-06-18T13:39:45.208 INFO:teuthology.orchestra.run.smithi179.stderr: 91.0% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-volume.log.gz 2024-06-18T13:39:45.219 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-volume.log 2024-06-18T13:39:45.220 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.cephadm.log: 89.2% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph.cephadm.log.gz 2024-06-18T13:39:45.222 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-18T13:39:45.235 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-client.rgw.foo.a.log 2024-06-18T13:39:45.248 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-18T13:39:45.263 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-client.rgw.foo.a.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-volume.log.gz 2024-06-18T13:39:45.275 INFO:teuthology.orchestra.run.smithi167.stderr: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-18T13:39:45.338 INFO:teuthology.orchestra.run.smithi167.stderr: 94.3% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-06-18T13:39:45.743 INFO:teuthology.orchestra.run.smithi167.stderr: 91.1% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mgr.y.log.gz 2024-06-18T13:39:46.215 INFO:teuthology.orchestra.run.smithi179.stderr: 92.9% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mon.b.log.gz 2024-06-18T13:39:46.480 INFO:teuthology.orchestra.run.smithi167.stderr: 92.7% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mon.c.log.gz 2024-06-18T13:39:47.624 INFO:teuthology.orchestra.run.smithi167.stderr: 91.8% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-mon.a.log.gz 2024-06-18T13:39:52.696 INFO:teuthology.orchestra.run.smithi179.stderr: 93.8% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.5.log.gz 2024-06-18T13:39:52.746 INFO:teuthology.orchestra.run.smithi179.stderr: 93.8% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.6.log.gz 2024-06-18T13:39:53.254 INFO:teuthology.orchestra.run.smithi167.stderr: 93.9% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.3.log.gz 2024-06-18T13:39:53.259 INFO:teuthology.orchestra.run.smithi179.stderr: 93.9% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.4.log.gz 2024-06-18T13:39:53.344 INFO:teuthology.orchestra.run.smithi167.stderr: 93.9% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.0.log.gz 2024-06-18T13:39:53.467 INFO:teuthology.orchestra.run.smithi167.stderr: 93.9% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.2.log.gz 2024-06-18T13:39:53.577 INFO:teuthology.orchestra.run.smithi179.stderr: 94.0% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.7.log.gz 2024-06-18T13:39:53.579 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-06-18T13:39:53.579 INFO:teuthology.orchestra.run.smithi179.stderr:real 0m8.402s 2024-06-18T13:39:53.580 INFO:teuthology.orchestra.run.smithi179.stderr:user 0m31.479s 2024-06-18T13:39:53.580 INFO:teuthology.orchestra.run.smithi179.stderr:sys 0m1.090s 2024-06-18T13:39:54.116 INFO:teuthology.orchestra.run.smithi167.stderr: 93.8% -- replaced with /var/log/ceph/3434f1e2-2d72-11ef-bc9d-c7b262605968/ceph-osd.1.log.gz 2024-06-18T13:39:54.119 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-06-18T13:39:54.119 INFO:teuthology.orchestra.run.smithi167.stderr:real 0m8.943s 2024-06-18T13:39:54.119 INFO:teuthology.orchestra.run.smithi167.stderr:user 0m36.407s 2024-06-18T13:39:54.119 INFO:teuthology.orchestra.run.smithi167.stderr:sys 0m1.248s 2024-06-18T13:39:54.119 INFO:tasks.cephadm:Archiving logs... 2024-06-18T13:39:54.119 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-17_21:08:03-orch-squid-distro-default-smithi/7760537/remote/smithi167/log 2024-06-18T13:39:54.120 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-18T13:39:58.904 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-17_21:08:03-orch-squid-distro-default-smithi/7760537/remote/smithi179/log 2024-06-18T13:39:58.905 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-18T13:40:03.417 INFO:tasks.cephadm:Removing cluster... 2024-06-18T13:40:03.417 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 --force 2024-06-18T13:40:03.610 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T13:40:03.928 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3434f1e2-2d72-11ef-bc9d-c7b262605968 --force 2024-06-18T13:40:04.121 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: 3434f1e2-2d72-11ef-bc9d-c7b262605968 2024-06-18T13:40:04.423 INFO:tasks.cephadm:Removing cephadm ... 2024-06-18T13:40:04.423 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-18T13:40:04.440 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-18T13:40:04.455 INFO:tasks.cephadm:Teardown complete 2024-06-18T13:40:04.456 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-18T13:40:04.465 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-18T13:40:04.465 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-18T13:40:04.482 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-18T13:40:04.566 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-18T13:40:04.575 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-18T13:40:04.575 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-18T13:40:04.578 DEBUG:teuthology.orchestra.run.smithi179:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-18T13:40:04.590 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-06-18T13:40:04.592 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-06-18T13:40:04.595 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-18T13:40:04.595 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-06-18T13:40:04.596 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 -337us[ -343us] +/- 59ms 2024-06-18T13:40:04.596 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 38 +852us[ +846us] +/- 63ms 2024-06-18T13:40:04.596 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 -650us[ -650us] +/- 75ms 2024-06-18T13:40:04.596 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-18T13:40:04.597 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-18T13:40:04.597 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-06-18T13:40:04.597 INFO:teuthology.orchestra.run.smithi179.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 -362us[ -347us] +/- 59ms 2024-06-18T13:40:04.597 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 +820us[ +835us] +/- 63ms 2024-06-18T13:40:04.597 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -673us[ -658us] +/- 74ms 2024-06-18T13:40:04.597 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-18T13:40:04.597 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-18T13:40:04.606 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-18T13:40:04.606 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-18T13:40:04.614 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-18T13:40:04.708 DEBUG:teuthology.orchestra.run.smithi179:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-18T13:40:04.777 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-06-18T13:40:04.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-18T13:40:04.805 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-06-18T13:40:04.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-18T13:40:04.833 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-18T13:40:04.842 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-18T13:40:04.850 INFO:teuthology.task.internal:Duration was 3020.416814 seconds 2024-06-18T13:40:04.851 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-18T13:40:04.862 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-18T13:40:04.862 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-18T13:40:04.864 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-18T13:40:04.901 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-18T13:40:04.911 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-18T13:40:05.261 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-18T13:40:05.262 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2024-06-18T13:40:05.262 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-18T13:40:05.324 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi179.front.sepia.ceph.com 2024-06-18T13:40:05.324 DEBUG:teuthology.orchestra.run.smithi179:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-18T13:40:05.348 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-18T13:40:05.349 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-18T13:40:05.366 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-18T13:40:05.416 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-18T13:40:05.416 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-18T13:40:05.433 DEBUG:teuthology.orchestra.run.smithi179:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-18T13:40:05.891 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-18T13:40:05.900 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-18T13:40:05.900 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-18T13:40:05.925 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-18T13:40:05.950 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-18T13:40:05.959 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-18T13:40:05.967 DEBUG:teuthology.orchestra.run.smithi179:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-18T13:40:05.988 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-06-18T13:40:06.015 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = core 2024-06-18T13:40:06.043 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-18T13:40:06.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-18T13:40:06.058 DEBUG:teuthology.orchestra.run.smithi179:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-18T13:40:06.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-18T13:40:06.099 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-18T13:40:06.108 INFO:teuthology.task.internal:Transferring archived files... 2024-06-18T13:40:06.108 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-17_21:08:03-orch-squid-distro-default-smithi/7760537/remote/smithi167 2024-06-18T13:40:06.109 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-18T13:40:06.171 DEBUG:teuthology.misc:Transferring archived files from smithi179:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-17_21:08:03-orch-squid-distro-default-smithi/7760537/remote/smithi179 2024-06-18T13:40:06.171 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-18T13:40:06.208 INFO:teuthology.task.internal:Removing archive directory... 2024-06-18T13:40:06.208 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-18T13:40:06.211 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-18T13:40:06.257 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-18T13:40:06.267 INFO:teuthology.task.internal:Not uploading archives. 2024-06-18T13:40:06.267 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-18T13:40:06.276 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-18T13:40:06.276 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-18T13:40:06.278 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-18T13:40:06.291 INFO:teuthology.orchestra.run.smithi167.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 18 13:40 /home/ubuntu/cephtest 2024-06-18T13:40:06.311 INFO:teuthology.orchestra.run.smithi179.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 18 13:40 /home/ubuntu/cephtest 2024-06-18T13:40:06.312 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-18T13:40:06.322 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-18T13:40:06.420 INFO:teuthology.run:Summary data: description: orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} duration: 3020.4168136119843 failure_reason: '"2024-06-18T13:05:38.388638+0000 mon.a (mon.0) 1238 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-06-18T13:40:06.420 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-18T13:40:06.499 INFO:teuthology.run:FAIL